Backtesting

Assessing your Model

Before investing some of your net worth into your trading bot, you might want to assess how much returns you might expect and how risky the investment is. Our approach is to measure performance at multiple stages. This involves keeping a part of the dataset as a holdout for validation and test performance:

The validation accuracy will be repeatedly measured when adjusting your strategy on the trained model. Independent of strategy, one can measure:

• Price direction accuracy across multiple prediction horizons. How often do you win, when long/short given position for given time span?

• Magnitude-weighted accuracy where large price changes are weighted as more important to get right.

When a strategy exists, you can simulate (backtest) your trades on the validation as well as on the test set to measure:

• Average 30 day returns given you start investing any time into the automated portfolio.

• Sharpe of the average returns, to see how stable those are (see the previous chapter).

Saving Backtests & Live Performance

Backtests are useful tools for assessing performance, but can still lead to false discoveries. Only if a model is actually deployed in the market for some time one can make any meaningful claims about its profitability. To do this, Robotter.ai saves every backtest result to Aleph.im and indexes the live performance of deployed bots. Thus, potential investors gain insight into the profitability and characteristics of a trading bot.

Aggregating the Best Robotters

The goal of Robotter.AI is to host the best data scientists and investors and support them in their journey towards their own AI-driven trading bot on Solana. We hope that in turn, they will help make Robotter.AI the best platform for training your own financial AIs.

Using our capabilities to standardize predictions and backtest, we gain the valuable opportunity to aggregate the best performing predictions into a meta-model, which will power the operation of our in-house hedge funds.

With continued development, we hope to find a trustless way to share predictions with other users, such that they can create their own meta-models, power their smart contracts with smart predictions and in turn develop new, data-driven financial products.