ARB430: Programming – building your own arbitrage software
One of the biggest constraints on arbing profit is time. So many arbitrage opportunities exist at the same time, yet as human arbers we can only execute one arb every two minutes. In addition, there are arbitrage opportunities that exist for just a few seconds, which is not enough time for a human arber to execute an arb.
That is why some professional arbers have taken to programming their own software to find, check and execute arbs automatically without human interaction.
Sports arbitrage lends itself to such automated systems (referred to as ‘bots’) because the arbing process is so mechanical and objective.
In this lesson we discuss the possibility of programming your own arbing bot, and the issues faced in such a proposition. For reference, while Sports Arbitrage Australia has never considered it, our research suggests that the costs of building such a system would be $200,000+ or the equivalent of two years’ worth of man hours.
There is a huge amount of work involved to be able to cover the many sports bookmakers, and match events, teams and betting lines across all bookmakers. In addition, for instantaneous analysis of the volume of data involved you will need state-of-the-art hardware and a lightning quick internet connection.
Basic features of arbing bots
– obtain odds from bookmakers – first things first, you need to obtain the odds from all bookmakers covered and store them in a database for future analysis. In the next section of this lesson we delve into the various methods for obtaining odds and the pros and cons of each.
– calculate and triple-check arbitrage opportunities – analysing the odds that have been extracted from bookmakers, arbitrage opportunities need to be identified and calculated. Arbs need to be discovered not just in single-markets but across betting lines too. Positive arbitrage opportunities then need to triple-checked to ensure no mismatched payout rules etc.
– automatically login and place the bets – capability to automatically login to each bookmaker’s website, navigate to the correct match page, place the bet on the correct line, and then confirm the bet. Any human involvement slows the arbing down considerably.
How to obtain the odds from bookmakers
There are three main sources for obtaining odds from bookmakers. Each is listed below and the advantages and disadvantages of each are discussed.
- bookmaker XML data feeds
By far the most common source of bookmaker odds for sports arbitrage bots is the XML data feeds that bookmakers provide directly from their website. Bookmaker XML feeds are free and importantly the bookmakers want you to use them. XML is a common programming language and therefore easy for an experienced programmer to create a bot with.
However, not all bookmakers offer XML data feeds for their odds, and those that do selectively limit the sports/leagues/teams/matches that are reported to the most lucrative categories for them. In addition, XML data feeds are slow at updating, sometimes the odds have changed long before the XML data feed is published.
Accordingly, XML data feeds are a good base to begin programming/testing a bot with, but not acceptable for serious programmers looking to have greater information with faster retrieval speeds.
- HTML scraping
The best option for programmers with ample time but limited funds is HTML scraping. HTML scraping literally refers to ‘scraping’ each and every bookmaker’s websites directly, much like loading the website manually and collecting all odds presented.
HTML scraping is far more rapid and reliable in odds collection because they are taken directly from the website in real-time, as and when the bot wants them – no more waiting around for an XML data feed to be published by the bookmaker. The odds are exactly what a human user would find when browsing the bookmaker’s website.
It is also free because the bot literally just browses the bookmaker websites.
However, a massive effort is required to setup a HTML scraping program and one needs to be setup for every bookmaker. Further, if the bookmaker changes their website layout or functionality, then the HTML scraper software needs to be updated to match the new website.
The biggest downside to HTML scraping is that it requires constant reloading of the bookmaker’s website for scraping new data. To keep up-to-date odds your bot needs to scrape the bookmaker’s website every 10 seconds or less. This is going to be discovered by the bookmaker and your IP address blocked very quickly. In order to use HTML scraping you need to either reduce the scraping refresh rate (not ideal) or delve into more complicated measures like IP rotation.
- third-party providers
A good option for programmers with adequate funds to invest is using third-party providers. In return for paying a monthly fee, a programmer can get a data feed of odds for any bookmakers, sports, leagues, and markets they want – no HTML scraping required, all the hard work is done for you.
However, these odds sources are not perfect either – they use the above two methods for obtaining odds and therefore have the same limitations. In addition, obtaining odds from a third-party provider means you lose any unique advantage that you might be trying to achieve because everyone else who uses the third-party provider has the exact same odds at the exact same time.
Defining sports/leagues/teams/events/betting lines
By far the most arduous task as part of programming your own arbing bot is matching the sports, leagues, teams, events, and betting lines from one bookmaker against another bookmaker. This is because each bookmaker has their own way of describing the bets. If we take Manchester United as an example, bookmakers can describe this team as:
- Manchester United
- Man United
- M United
- Man U
- Manchester U
- Manchester U.
- or some other abbreviation
Because all of the above descriptions relate to the same team, then the bot needs to be able to refer to a central mapping table whereby the different descriptions get referenced to one unique code, in order to identify arbitrage opportunities. Only by collecting odds against a unique code will your arb-finding logic be able to function.
To make matters worse, some bookmakers even have multiple versions of a team’s description depending on the betting line.
While Sports Arbitrage Australia has never considered programming our own bot, our research suggests that the arbing with custom bots does produce a greater overall profit from sports arbitrage, but the bookmaker limits come much faster.
The bots are placing so many bets with each bookmaker that they will be identified quickly, and the higher limits are said to last four days maximum when arbing through a bot (instead of two weeks when arbing manually). However, because the bot is executing such a large volume of arbs then the profit generated in that four day period is considerably higher than manual arbing.
Another benefit of bots is that when bookmakers restrict betting limits to small amounts the bot will continue arbing, making $1 profits consistently, where a human arber may get frustrated once their betting limits are restricted.
Anecdotal evidence suggests that in time, even the tiny betting limits imposed on arbers running bots are further reduced. Often to $20, then $5, $2, $0.20 finally $0.00!