line scraper for Pinnacle odds

**Busterflywheel** · 01-20-10, 12:32 PM

Ill pass..thanxx

**runnershane14** · 01-20-10, 04:28 PM

thanks arwar good stuff!

**pats3peat** · 01-20-10, 06:08 PM

Pretty sweet, so this automatically updates the Excel/Notepad files whenever i run the scraper?
P.S is there anyway i can make it so I dont get the NCAAb lines (only the NBA lines)

good tip on the windows scheduler

**arwar** · 01-20-10, 10:01 PM

Originally posted by pats3peat

Pretty sweet, so this automatically updates the Excel/Notepad files whenever i run the scraper?
P.S is there anyway i can make it so I dont get the NCAAb lines (only the NBA lines)

good tip on the windows scheduler

i knew somebody was going to ask this...how about if i put another link with NBA as default checked?

**arwar** · 01-20-10, 10:02 PM

Originally posted by Busterflywheel

Ill pass..thanxx

pass it on..............

**arwar** · 01-20-10, 10:13 PM

Originally posted by pats3peat

Pretty sweet, so this automatically updates the Excel/Notepad files whenever i run the scraper?
P.S is there anyway i can make it so I dont get the NCAAb lines (only the NBA lines)

good tip on the windows scheduler

i'll put another link with NBA as the default??

**pats3peat** · 01-21-10, 02:43 AM

Im all set , i was just seeing how it works.
Are you going to use it yourself... this would just be useful for looking at how odds change like every 30 minutes, etc right?

**zorze** · 01-21-10, 01:00 PM

really nice stuff, thank you..

**Busterflywheel** · 01-21-10, 05:19 PM

Not so bad afterall..Must admit I was scared of a virus or something...THX though..

**rfr3sh** · 01-22-10, 04:12 PM

**rfr3sh** · 01-22-10, 04:16 PM

i dont know if its asking too much to add soccer aswell..no biggie if you cant

**pats3peat** · 01-22-10, 06:11 PM

what are you guys using this for? Is looking at line movement changes that important to see incrementally?
anyhow, cool programming. and people obviously like it/are using it.

**JoeVig** · 01-22-10, 07:29 PM

Won't they get mad if you sit there and hammer their page? I would think everyone is sensitive with all the DDOS attacks going around.

**durito** · 01-22-10, 07:42 PM

Originally posted by JoeVig

Won't they get mad if you sit there and hammer their page? I would think everyone is sensitive with all the DDOS attacks going around.

Pinny has always been sensitive. They will ban your ip.

**JohnAnthony** · 01-22-10, 09:22 PM

Can we ask them? I mean, can we ask them without getting banned or flagged? just ask, just ask..

And thanks for your regardless, arwar!

**pats3peat** · 01-22-10, 09:43 PM

Originally posted by JohnAnthony

Can we ask them? I mean, can we ask them without getting banned or flagged? just ask, just ask.. And thanks for your regardless, arwar!

No point in asking because they prob have software that bans for IPs for retrieving their lines like every 2 seconds

**icsky3** · 01-22-10, 10:20 PM

Good stuff there

**blix177** · 01-23-10, 02:04 AM

If you refresh too quick they will ban your IP. They assume it is a DOS attack.

**dkp** · 01-24-10, 10:48 AM

Thanks... good stuf..

**arwar** · 01-25-10, 02:14 AM

okay, I don't know if I can hit all of the issues but i'll try.
First regarding hitting the site too often: when I was writing the scraper to scrape SBR, the testing phase pounded the site for pages - I would scrape maybe 100 pages in a minute or so. I was always looking (expecting?) that I might at least get blinked - but nothing ever happened. (aside I see that Vegashotsheet.com lnked directly to SBR this past season). A denial attack would come from 100's if not 1000's of boxes at the same time. With today's bandwidth a single PC is more like a gnat. So long as the only activity is accessing publicly available pages it should be a non issue.

A few people are concerned about trojans,etc. There aren't any. I don't have time or inclination to go the script kiddie route.

I have updated the Pinny line scraper (haven't released it yet). Rather than using the Windows scheduler, i just added a timer so you can scrape at intervals from 1 minute to daily. I let it run scraping every minute for 36 hours and ended up with a whole lot of useless data. But it is there and I will upload it shortly,

**arwar** · 01-25-10, 02:15 AM

Originally posted by rfr3sh

i dont know if its asking too much to add soccer aswell..no biggie if you cant

soccer lines from Pinny?

**arwar** · 01-25-10, 02:16 AM

Originally posted by pats3peat

what are you guys using this for? Is looking at line movement changes that important to see incrementally?
anyhow, cool programming. and people obviously like it/are using it.

I was wondering that myself, but i think we can improve on it.

**arwar** · 01-25-10, 02:39 AM

scraping scoresandodds.com

okay. I was running a query Friday nite and realized that I needed historical data on NBA money lines. covers.com has it buried way down but it is a pain in the ass to get too, so I remembered scoresandodds has that as well line movement histories in a condensed (a good thing) form. Also I seem to recall another post in another thread complaining of the difficulty of scraping data from their site. so necessity being the mother, etc. etc. I wrote a scraper for S&O. I ran it from 01/02/09 until today. the attached text file contains that data. For some reason, that I will investigate later, baseball got filtered out. But my NBA lines are there, as are NFL, NCAAB and I guess WNBA. The text file attached has that data. It is a .txt file that will open in Notepad by default. If you change the extension to a .csv file it will then open by default in EXCEL. I haven't done any refining on this data, but it is still useful and there is a lot of it. I don't know whose lines they are, but there they are. I will post this scraper as well. It has a feature so you can define any date range you want. The range i ran 01/02/2009-01/24/2010 took just over 10 minutes and dumped out the attached file. This stuff gets cool because you can study things like streaks during certain months. I will fix it for baseball and add some functonality so you can scrape a random sample, say 100 randomly selected games in July - or whatever. I am also working on building a database combining all of the data from different sites into a central clearinghouse. All you have to is learn how to write an SQL query but if you have ACCESS with your EXCEL, it will do it for you....

happy scraping

**Netko** · 12-27-10, 07:10 PM

hello, i cant find scraper on http://www.parlaysrus.com/
Can someone send me scraper or whera i can download scraper?
thx in advancefor help

**pedro803** · 12-27-10, 11:01 PM

Wow Arwar, thanks! You are the bomb!!

**Netko** · 12-28-10, 01:30 PM

Originally posted by pedro803

Wow Arwar, thanks! You are the bomb!!

can you tell me from where u download this line scraper? thx

**Konkss** · 12-28-10, 01:50 PM

i'm not finding the scraper to, can anyone post the link ?

**aggieshawn** · 12-28-10, 02:04 PM

bet the ponies today

**mikew** · 02-17-12, 12:01 AM

bump, anyone have the link

**Sportsguy_USA** · 02-17-12, 08:38 PM

Ummm...why do you need the scraper? They offer an XML feed API for their match ups and odds covering all sports. I consulted with a group who used it, among others, to develop a social Facebook sportsbook (entertainment purposes), but was a fairly cool idea - bet against friends on facebook, etc.

**goty0405** · 02-17-12, 08:43 PM

Originally posted by Sportsguy_USA

Ummm...why do you need the scraper? They offer an XML feed API for their match ups and odds covering all sports.

This.

I use it and it's easy and they wont ban your IP for being a dumbass and hitting their normal page servers too much.

**strixee** · 02-18-12, 03:06 AM

Maybe he's interested in buying points.