After a week delay while the United States elected an authoritarian misogynist xenophobe (more on that in another post), CCP games will be releasing one of the most fundamental player-focused changes in EVE Online; free-to-play in the form of Alpha Clones. This is a critical inflection point in the history of the game I first picked up in 2003, as an alternative to Earth and Beyond MMO which was on its downward spiral. Either the F2P option will be ignored and EVE will not grow, or EVE will grow faster than ever before.
A small trip down memory lane
CCP has been able to keep EVE Online around for over 13 years, which is an absolute eternity in video games - EVE is older than World of Warcraft, and numerous other high profile MMO games have come and gone in the intervening decade.
For 11 of those years (domain registered December 2005), eve-central.com has been around in some form or fashion to provide an easy way to browse and make sense of EVE’s regional segmented market. The first version of EVE-Central forced users to button smash an in-game button in the market UI.
A simple little wxPython application scanned for CSV files, and simply POSTed them to a webserver backed by a single PostgreSQL database. The entire thing was a pile of Python on a single dedicated server. A simple market browser, along with a buy to sell tradefinder were quickly rolled out in 2006. The underlying code is terrible and also stable - it was last touched sometime in 2007!
EVE-Central grew an API (XML!), which has continued to be its largest source of traffic. Most popular is the “marketstat” API which gives a digested price snapshot based on a few filtering operators (regions, types, etc).
EVE-Central quickly outgrew the original dedicated server and moved to a much beefier co-located server (at Herakles in Sacramento). The Python stack was still running, caching was being added, and the upload client mostly hummed along with nearly no intervention. Around this time EVEMarketData launched, and provided an alternate UI. More importantly, EMD jumped the gun and came up with a way to scrape cache files that the EVE client wrote to disk, sidestepping the entire annoyance of manually exporting data to disk. Not to be outdone, the EVE-Central client gained this support around 2009.
As the API was by far the most computationally expensive operation in the whole system, it was used as a learning vessel and was re-written in Scala with Akka as the framework in 2012.. This proved to be quite a bit more scalable, but EVE-Central moved co-location providers to Hurricane Electric in Fremont, as bandwidth costs were becoming prohibitive in the original arrangement. The API calls to get the entire market contents dominated the data rate, and to date can peak at several thousand calls a second.
EVE-Central is now on four servers, a pool of two replicated PostgreSQL machines, one main application server which refuses to die, and one CREST scraping server. All of them are 2010-2012 vintage, and a number of disks or other systems should have likely failed at this point. It keeps on trucking, and breaks even on AdSense revenue vs the co-location cost (but not initial server investment).
EVE-Online is somewhat unique in the game space since it has always had some form of API access to various game features. EVE-Central existed and became popular to fill the gap for market information access, and provide tools and a user interface on top of the market data.
When CCP introduced CREST, the first API with market data, we stopped supporting user uploaded data and the EVE Market Data Relay system. User data always had the problem of market manipulators - without a good trustworthy signing system, it was trivial to upload old/different/fake data to EVE-Central and the EMDR relays which would be disseminated. A number of hidden features never open-sourced dealt this problem in relatively trivial ways. With CREST in the picture, user sourced data was no longer needed, and the chapter of cache scraping and maintaining a client application came to a close.
Thankfully CCP now provides not only one but two (why not both!) APIs to directly access market orders from outside of the game - CREST and ESI. This is fantastic - the data is fresh, and directly from the source. The need for EVE-Central to republish exact market goes away, except for a few legacy use cases.
EVE-Central was also extensively built around a push model - data was pushed into the system from clients, which causes us to spend a lot of cycles and round-trips sending a near constant several hundred RPS to CCP’s CREST server, often for data which is never consulted by users of EVE-Central.
So, where does EVE-Central go from here?
EVE-Central in the age of Ascension
EVE-Central has done a poor job of adapting and building tools which enhance the player experience, and this is a disservice to the new players which the robust EVE third-party tool community serves. However, it still provides a strong utility to “spot check” the market, and is a great data source to layer tools which should have been written in the intervening 12 years for market research and logistics (such as a trade finder, market research, and numerous other tools).
I want to help change this, and in order to go there I will be making several key changes to the way EVE-Central operates.
-
Deprecate or discourage use of “low-value-add” APIs. CREST and ESI provide endpoints to get market orders and statistics directly. There is no reason to ask EVE-Central for this data. APIs such as
quicklookfall into this bucket. These APIs also generate the most traffic (excluding those called by the site its self). By end of year, I would like to turn off the current XML basedquicklook. -
Change EVE-Central to a pull model The core tools of EVE-Central will operate in a pull-then-cache model, for when we need the view of the universal market in order to operate several of our tools. I started support for this in a CREST world, and will be building on these tools in a ESI world.
-
Provide higher level APIs and UIs As
marketstatandquicklookend up sunsetting, provide higher value APIs which can provide more targeted answers, and build UIs to drive them. Part of this effort is introducingECQL, a query and computation language for the market, which lets you ask questions and combine data in new ways. -
Enrich the core tools, like the tradefinder Enough said.
-
Move off dedicated hardware This is a personal sanity issue - if EVE-Central were to suffer something like a failed disk, it would be several days before I could get a replacement in - and those database SSDs currently are on a death watch in terms of write rates. While a number of facets of “the cloud” are more expensive (bandwidth, etc), the capital investment needed to keep the current EVE-Central running aren’t worth it in the long run. EVE-Central can also get a lot closer (same datacenter) as the new ESI endpoints, which cuts down on a lot of our transfer costs and improves latency significantly.
Look for some subset of api.eve-central.com traffic to be running
off of Google Compute Cloud in the coming weeks, as a validation
test. The move on the front-end won’t be fast, but this is by far the
oldest code in the system and needs to be replaced sooner than later.
Best of luck to CCP in their adventure with F2P, and I hope EVE-Central will be there to support the players of EVE, even if I mainly login to queue skills and spin my ships in a station.