Skip to main content

Web mining structure and challenges

Short answer

Web mining applies data mining to discover patterns from web content, structure, and usage. Content mining extracts knowledge from page text and media, structure mining analyzes hyperlinks, and usage mining analyzes logs and clickstreams. Challenges include scale, dynamism, heterogeneity, noise, link farms, user identification, privacy, multilingual data, and evaluation.

Describe web mining along with its structure. What are the challenges in web mining?

Web mining is the use of data mining techniques to automatically discover and extract useful information and patterns from web documents, hyperlinks and usage data of the World Wide Web.

Structure (taxonomy) of web mining

               Web mining
     ____________|______________
    |            |              |
 Content      Structure       Usage
 mining        mining         mining
 (page text,  (hyperlink     (server logs,
 images,       graph:         clickstream,
 video)        PageRank,      sessions)
               HITS)
  • Content mining: knowledge from the content of pages (classification, clustering, extraction, summarisation).
  • Structure mining: analysis of the link graph to rank pages and find communities.
  • Usage mining: discovery of user navigation patterns (association rules, sequence patterns, clustering).

Challenges in web mining

  1. Huge size and rapid growth of the web: scalability of algorithms.
  2. Dynamic nature: pages change constantly or disappear; new pages appear.
  3. Heterogeneous and unstructured data: HTML, text, images, video; no uniform schema.
  4. Noise and poor quality: spam, duplicate, advertising and misleading pages; irrelevant data.
  5. Link and anchor-text issues: spam links, link farms.
  6. Usage data issues: identification of users and sessions (proxies, caches, dynamic IPs), incomplete logs.
  7. Privacy and security concerns of tracking users.
  8. Multilingual content and semantic understanding.
  9. Evaluation difficulties: lack of ground truth.

✓ Answer checked by Claude Opus 5.5 (high effort)

Related questions

All Data Mining chapter 7 questions →