MMO P&S LinkedIn Data Collector

[SelfDev]

I tried to gather data from LinkedIn for a test, but after a little while, my requests stopped working. I think it’s because of a captcha or something similar. When I logged in using a spare LinkedIn account, I got a message about using a bot. But I know lots of people still collect data from LinkedIn.

How do they do it?

 

[Chonessah]

How’d you gather stuff from LinkedIn?

 

[SelfDev]

I made it pretty easy. Logged in with my own info, searched for what I wanted, clicked on each profile, and grabbed the stuff I needed. I used tools like Selenium and Scrapy to help.

 

[Chonessah]

I made it pretty easy. Logged in with my own info, searched for what I wanted, clicked on each profile, and grabbed the stuff I needed. I used tools like Selenium and Scrapy to help.

Websites can spot Selenium pretty fast now because it’s kinda outdated.

 

[SelfDev]

Websites can spot Selenium pretty fast now because it’s kinda outdated.

What else should I try? How can I stay hidden so websites don’t notice me?

 

[Chonessah]

You could check out Playwright. I’ve been using it to do stuff on Google, and so far it’s working really nice.

 

[SelfDev]

You could check out Playwright. I’ve been using it to do stuff on Google, and so far it’s working really nice.

Hmm, that sounds cool. I didn’t really check it out fully yet—just searched it real quick. What makes Playwright better than Selenium?

 

[Chonessah]

Hmm, that sounds cool. I didn’t really check it out fully yet—just searched it real quick. What makes Playwright better than Selenium?

I like it better than Selenium because you don’t have to set up a special driver, deal with version stuff, and it’s a little easier to use.

Most importantly, it is a new framework that is a little less likely to be noticed by websites.

 

[Landrater]

Puppeteer is a handy tool that can run a browser in the background. Have you tried using proxies?

 

[Chonessah]

Puppeteer is a handy tool that can run a browser in the background. Have you tried using proxies?

Puppeteer and Playwright both control a web browser in the same way. Some browser tools add extra settings for privacy, and you can test from different IP addresses with them. Have you tried that?

If you want a different tone or need it for a lawful testing project, tell me and I’ll adapt it.

 

[SelfDev]

Puppeteer and Playwright both control a web browser in the same way. Some browser tools add extra settings for privacy, and you can test from different IP addresses with them. Have you tried that?

If you want a different tone or need it for a lawful testing project, tell me and I’ll adapt it.

How do you set up special browser tools for testing, and how do those work with LinkedIn when you’re just checking things?
I tried using proxies before, but something went wrong and I accidentally paid about $9 in 6 seconds. :/

And a couple of safe tips (5th‑grade level) about proxies and costs:

Use a provider that gives test credits or a small trial so you don’t spend a lot at once.

Turn on a spending limit or check the dashboard often so you can stop it fast.

For testing LinkedIn, use LinkedIn’s official tools or APIs and follow their rules to stay safe.

 

[Landrater]

If you use scrapers like Selenium, your account can get blocked!

It’s smarter to write your own program that makes the browser act like a real person so it can pass LinkedIn’s checks.