Game Oracle v3.3 Is Live!

TLDR - New Map, New Data, and Better Search Capabilities

The latest version of Game Oracle just shipped and it packs a punch with some major updates to our tools:

• New Model: We have a new model and it delivers a 2% increase in accuracy when searching for similar games (e.g. finding competitors) and finding markets using descriptions (e.g. validating a new idea) in Data Explorer
• New Map: The updated model means we have a brand new Steam map! We've also improved our saturation scores and improved the usability of Saturation Map 
• Game Gap Renovation: We've completely overhauled Game Gap — a new clustering algorithm that now identifies market segments with under served audiences and missed opportunities, plus a new outlier section
• New Improved Data: We've improved the reliability of our sales estimates and introduced AI summary data meaning you can now filter games by AI disclosure and explore AI statements

We also fixed some bugs and issues with this release:

• Stopped the lock symbol from mistakenly appearing over some tools even when subscribed and logged in
• Some games were not searchable from our dashboard search bar because of how non-alphabetical characters were being handled, this has been corrected
• Prevented some long game titles from breaking our page layouts (looking at you Planetenverteidigungskanonenkommandant)
• Histogram bin sizes were bugging out with some data - fixed and introduced custom size

We hope you enjoy the updates! Please tell us if you notice any issues. Keep reading to see the details and what is coming up next!

New Model, Better Search

Our core USP at Game Oracle is our map of Steam -  a mathematical model that describes the similarity between games. You can achieve a lot with that core functionality: you can describe your idea in plain english and surface existing games adjacent to the idea for study, you can quantify and visualise market saturation, and you can cluster market segments to identify under served audiences and missed opportunities.

So for us, this model is a big deal, which is why we work so hard behind the scenes to try and make it as accurate as possible. We have used a combination of machine learning and our own blood, sweat, and tears (mostly tears) to curate a large dataset of tens of thousands of games with positive and negative pairs  — a positive pair are two similar games (based on themes, genres, mechanics, and art) and a negative pair are dissimilar games. 

We use this data to try and help our model understand what makes games similar or dissimilar. One of our key learnings, which has helped shape our latest model, is the important of game mechanics for shaping what makes games similar across the wider market place, but then artistic style should help shape what defines similarity locally i.e. between neighbourhoods of genres, sub-genres etc. 

Using our carefully curated test data, we rigorously assess our model and make sure it understands what makes games similar or dissimilar to one another. Our newest model does a great job at this. When searching for similar games in our testing data, search accuracy has increased from 94.8% to 96.7%. It might not seem like much, but it can make all the difference when trying to build confidence around your market research.

Grab Your Compass, We Have A New Map!

The new model means we have a new Saturation Map, the 2D heatmap that allows you to not only visualise market saturation across the whole of Steam but also interactively explore that map. 

We've also introduced improvements to how you interact with the map to help address an issue with offering overly precise navigation. Now, I know that sounds counter intuitive but let me explain. When we create the 2D saturation map we're constraining our enormous mathematical model (which contains over 2000 dimensions!) into a 2D space. That involves compressing a lot of data down and unfortunately it is impossible to achieve this without some information loss (this is a really interesting open-research question and for those interested I suggest reading about dimension reduction, namely this review by Encord or this review in Nature). We've spent a long time making sure we prevent as much information loss as possible but there will always be some.

Ultimately what this mean is, on the map games are generally placed closer together if they're more similar and further apart if they're very different. However, the similarity of games compared to the search results you get in Data Explorer or Game Gap is less accurate. So interpreting the games in Saturation Map that are immediate neighbours to each other as the most similar games in the market is misleading, but selecting large regions of the map based on saturation is actually remarkably accurate. We have therefore restricted the interaction with Saturation Map so you can only select a minimum region of several hundred games. This ensures that when you select a region you're definitely selecting a market of similar games and the summary and analysis of that market is backed by sufficient data.

Game Gap Renovated!

We have complete renovated Game Gap by introducing a new clustering algorithm and separating the tool into three powerful sections. In Game Gap we use our SOTA model to identify market segments — we find thousands of indie games that have performed well and have low saturation scores, then we use a bespoke neighbourhood search algorithm to construct market segments around those games. We then filter those market segments into two key categories:

1. Underserved Audiences: groups of similar games with below average market saturation, the absence of large competitors, and proven player demand — these market segments present an opportunity to deliver a high quality experience to an existing audience that is possibly underserved.
2. Missed Opportunities: groups of similar games with proven player demand but with few (if any) highly rated games — these market segments present an opportunity to deliver a high quality experience to an existing market that is possibly lacking great games.

We then allow our users to filter segments for their particular interests. By selecting and exploring segments you get a detailed analysis of the market segments, including the games contained within the segment, an overview of the art styles and game mechanics, a summary of the reviews and what players love/hate, and an overview of the opportunities within that segment.

Finally, we have also introduced an Outliers tool within Game Gap. There is a lot of noise in Steam data with most games being average and not pushing boundaries. We believe a lot of the signal is in the outliers, which we define as games with exceptionally low saturation scores but a high number of estimated sales and good reviews. We have created a detailed overview of the top outliers that we hope can provide inspiration for more unique and creative game ideas. You can explore these outliers, inspect summaries of their key features, things players loved/hated, and potential opportunities for similar game ideas.

Better Sales Estimates & AI Disclosure Data

We're always trying to improve our underlying data and in this release we have made two critical changes that we feel will improve reliability and more meaningful search results:

• Improved Sales & Revenue Estimates: sales and revenue are estimated using the Boxlieter method, but this is dependent on reviews on Steam. Previously we were using all the Steam reviews but the issue is, not all Steam reviews originate from Steam purchases. This means we cannot guarantee the price the game was purchased at or whether it was purchased at all! A lot of reviews come from users that have received the product for free which can cause misleading sales estimates. We have now updated our statistics to exclusively use reviews from confirmed Steam purchases when estimating both sales and Steam revenue.
• AI Disclosure: the game development landscape is changing rapidly in response to generative AI. In response to feedback from our customers, we are now collecting the AI disclosure statements for games on Steam and now you can filter results for those that disclose the use of AI. You can also view those disclosures yourself in your search results.

What To Expect In v3.4

Developing v3.3 has kept us very busy over the past month but we have much more to come! We're now turning our attention to v3.4 which is going to introduce some exciting new updates:

• Introducing Review Summaries for every single game on Steam to help you understand what players loved/hated about each title
• New timeseries data including Wishlist-to-Sales conversion ratio estimates
• Introducing Performance Navigator — create your own custom dashboard for our game idea and we'll forecast wishlist, sales, and revenue performance, help you set key performance metrics and objectives, suggest data driven pricing and marketing strategies, and help you craft the perfect Steam page

If you want to stay up-to-date with all these changes and be the first to know when new products launch you can subscribe to our free monthly newsletter below.

Enjoying these insights? Get the latest Indie Innovations in your inbox every month