stopped working - no more items left to scrape - (no results)
no more items left to scrape - no results
all IG accs fine, no PV's or anything.
All IG accs created on same IP and all accs logged in fine without me needing to add cookies
Even logs in fine using Auto login Chrome
but this error happens?
![](https://www.host-stage.net/images/affiliates/hoststage-aff-2-728x90.jpg")
Comments
it usually gives me about 50-60 boxes where i have to click OK. There are 250+ scrapers
What happens when you use 1 account only?
I will have to check the program and see if everything is fine or changed from IG.
I will respond back soon.
Regards,
AD
IG changed and added new type of acc -
is_professional_account
Program checks before hitting the API points for safety and less use of API data. Bad news is both
is_business_account
is_professional_account
can have email data - so had to fix this.
Working fine now.
More features incoming
Please download the latest version 3.36
Regards,
AD
accounts> @Tasker18Admin said:
Not with 1-5 accounts. It just says API limit reached, but there are no results. i am scraping emails from usernames.
I noticed recently i am not able to get as many results from the same number of scrapers. not sure why. all cookies are OK and accounts logging in fine
P.S. I noticed IG has changed their Phone Verification layout yesturday, there is currently a bug where you cannot confirm code from phone number. even though this is not related, it tells us there has been an ig update
DM me and i can give VPS details if need be, unless you are happy to test on your end
oh sorry didnt get your message - didnt reload page this morning before replying, i'll try download and see if it fixes
update - not working much still. I noticed in logs-
"We're having trouble connecting right now. Please try your donation another time."
With 250+ accounts, only able to get a few hundred emails a day
It still seems to be happening (see image)
Attached log of an account doing this
see here as well
It was working last when?
You should try with 1-2 account rather than 250+
Can you do that?
Checked your logs.
You have to understand a few things before scraping. YOU HAVE TO HIT THE BUSINESS PROFILES Only not random profiles. The quoted log does not have a single email address in it if i searched correctly. Not sure what is you API usage limit but what ever it is - is fully wasted!
The email scraper / parser is 100% OK as if works for 1 will work for 10000 which we know.
So, before loading the usernames - filter them by business. IT will reduce the disappointment
The email scraping / parsing had issues which is fixed as you know it scrapes them BUT the program has no ability to scrape email from an account which is not providing email address in it's email field.
IG - it is becoming a mess due to it's high expectation of money.
log of 1 example acc - API limit was 50, only managed to get like 2 results
Program did 2 -3API calls and 5-6 normal calls.
It has 2 email address.
What is wrong with this?
Do you expect more emails?
IF yes then are you sure the imported users have public_email in them?
I think not ( by the source is see )
What do you think?
OK no problem, i will test some more
The thing is, yesturday i got 900 emails successfully (from about 2000 usernames) with 250 scraper accounts max 25 API calls. Since the daily limit is 50, after that I ran again and it only got like 60-70 emails
That was your lucky day - the accs had email in them but then the feature got broken.
Then is fixed.
Now you should be like before - as i said before, the situation is same like before as i see. IF the program does not do it's job then is an issue.
If is business but does not scrapes then issue
If has email but not getting it then is issue
But please do test on small scale and then run mass.
Looks like you are doing ok with API limit feature, that is a nice one right?
Yes 7-15 break with 50 API call has worked great!
Yes I think you may be right, many users in that list had no email - it seems OK today
So if i have a raw username list, how is it best to get business account usernames only? By the time I have used your software to get "Is Business Account", wouldn't that mean I have the emails already and can just export?
What i mean is - is there a halfway point between Raw Usernames -> Get Emails
Apologies if you have explained this before - it looks like any kind of scraping for targeted information gives EVERYTHING (email,phone,city,isbusiness,etc)
gives EVERYTHING (email,phone,city,isbusiness,etc)
Good news then!
The profile details scraper is what you need.
Load the usernames, proxies and scrape them.
Then filter out the only business accs + not private
( i am going to add more columns in that wizard so you get what you want -for filtering )
Then hit them to get mails. Simple and straight forward
I have explained this before in PM and here again. No worries as softwares are different but users are same. Sometimes it is hard to adapt.
You need profile details scraper + proxies.
Regards,
AD
Thanks
For some reason it says i can only use 1 account for Profile Details Scraper? How come, this will be too slow
Do this
Check an acc
Open the profile details scraper
Import proxies
Import users
Then scrape. It will not use the account but the proxies only.
Make sure the proxies are rotating.
If need i will tweak it. No worries.
Regards,
AD
Yes, i have done this. Although it is still quite slow still. When scraping users via big pages, most of them arent business account so it only is filtering 1 user every 5 seconds
Could you enable Profile Details Scraper to use all accounts? I can then use 2,000+ scrapers to simultaneously filter business accounts
1 userevery 5 seconds even if not business acc*, so not scalable at the moment
PROFILE DETAILS SCRAPER DOES NOT USES ACCOUNTS.It is dummy.
Only 1 account check is sufficient.
I will add multi threading option - but you'd need rotating proxies as ig will block the IP after 13-15 fast request.
No worries. Next version will allow you to do it full speed.
You will run out of IP i am sure. Do not blame me then.
ok thank you i understand !
Please check the new version
Video explanation may help.
Regards,
AD
Thanks! Just trying this out now, going to get some rotating proxies
You recommend 2 minute rotation
Is 4G rotating necessary for this, or can i just use ipv4 residential or even ipv6 Datacentre?
Is there any difference in terms of limits do you know? As of course ipv6 DC is the cheapest
IG forbids - extensive request.
This is the point of everything we discussed. You need to avoid same IP while scraping.
Yes, everyone wants cheap but it is totally on you if you want to waste your time by trying spammed data center proxies or speed up your process by using fast rotating mobile / residential IPs.
I am sure you'll be adjusted upon experience.
Regards,
AD