Public data vs personal data in web scraping

· 3 min read · CrawlMine guides

This is not legal advice. It explains common concepts. Ask a qualified lawyer about your specific project.

Two different questions

When people ask whether they can scrape something, they often mix two questions:

  1. Is it public? Can anyone see it without logging in?
  2. Is it personal? Does it relate to an identifiable person?

They are independent. A product price is public and not personal. A private customer list is personal and not public. A name and phone number on a public listing are both public and personal, and that combination is where most of the legal care is needed.

What counts as personal data

Personal data is information about an identifiable individual. Obvious examples are names, email addresses, phone numbers, photos and home addresses. Less obvious ones include usernames, profile links and combinations of details that together identify someone.

Laws such as India’s Digital Personal Data Protection Act, 2023 and the EU’s GDPR regulate how personal data is collected and used. Being visible on a public website does not automatically take data outside these laws. The DPDP Act has an exemption for personal data a person has made publicly available, but how far it reaches is still being clarified. The GDPR has no general public-data exemption.

Business data is usually different

Information about businesses, rather than people, is generally not personal data:

  • Company names, categories and addresses
  • Product prices, specifications and stock
  • Property listing details such as size and price
  • Job posting details such as title and skills
  • Published rates and tariffs

That is why most commercial scraping projects can avoid personal data entirely. They need facts about products, places and markets, not about individuals.

Grey areas

Some cases need thought:

  • Sole traders, whose business name is their own name
  • Reviews, which contain text written by people and may include names
  • Listings by private sellers, such as used cars or property, which may show contact details
  • Job postings that name a recruiter

The practical answer in each case is the same: collect the business information and leave out the fields that identify a person, unless the project has a documented lawful basis to include them.

Practical defaults

  • Exclude personal fields by default. Drop names, phone numbers, emails and profile links during collection, not afterwards.
  • Do not collect behind logins. It changes both the legal and ethical picture.
  • Minimise. Collect only fields the use case needs.
  • Document. Record what you collect, why and from where.
  • Check before including personal data. If a project needs it, confirm the lawful basis, purpose and retention first.

These are the defaults we use on every project, from business listings to reviews.

Why it matters commercially

Beyond legal risk, personal data creates operational burden: storage rules, access requests, deletion requests, security obligations. A dataset without personal data is simpler to store, share and analyse. For most business questions, it loses nothing.

Summary

Public does not mean free to use when the data is personal. Most business uses of web data do not need personal data at all, so leave it out by default and check carefully when a project truly needs it. Read more in is web scraping legal in India. For a project review, send us your sources and fields or email [email protected]. See also our web scraping services.

Talk to us directly

[email protected]

Send the websites and fields you need. A person on our team reads every message.

Email us

Send us the sites tonight. Data by morning.

Tell us which websites and fields you need. We confirm what is practical, how it should be delivered and what it costs.

  • Goes straight to [email protected]
  • Reply within one working day
  • A sample or pilot before any recurring plan
  • NDA available before you share details

[email protected]