CppBacktester

Program Execution Flow

This document provides a detailed walkthrough of the execution flow in CppBacktester, from initialization to final results. Understanding this flow will help you better utilize the framework and implement custom strategies.

High-Level Overview

Here's the high-level flow of a typical backtesting session:

1. Load configuration from config.json
2. Create BacktestEngine with loaded config
3. Load market data via DataLoader
4. Create and initialize a Strategy
5. Run the backtest loop bar by bar
6. Calculate and output performance metrics

Note: Flow diagram image not available

1. Configuration Loading

The backtester begins by loading configuration settings from config.json:

// In main.cpp
Config config;
std::string configFilename = "config.json"; 
if (!config.loadFromFile(configFilename)) {
    Utils::logMessage("Main Warning: Proceeding with internal default configuration.");
} else {
    std::cout << "Configuration loaded successfully from " << configFilename << std::endl;
}

During this phase:

• The Config class parses the JSON file using the nlohmann JSON library
• Default values are applied for any missing configuration parameters
• Configuration validation is performed
• The loaded configuration is stored for later use

2. BacktestEngine Initialization

The BacktestEngine is created with the loaded configuration:

// In main.cpp
std::unique_ptr engine = std::make_unique(config);

During engine initialization:

• The engine stores a copy of the configuration
• A Broker instance is created with parameters from the configuration
• Initial variables and data structures are set up

3. Data Loading

Next, the engine loads market data:

// In main.cpp
if (!engine->loadData()) {
    Utils::logMessage("Main Error: Failed to load data. Exiting.");
    return 1;
}

The data loading process:

1. BacktestEngine creates a DataLoader instance to manage loading process
2. DataLoader checks configuration to determine data source:
   • If SourceType = "CSV", a CSVDataSource is created
   • If SourceType = "API", an APIDataSource is created
3. For CSV data:
   1. CSVDataSource opens the CSV file specified in INPUT_CSV_PATH
   2. CSVParserStep parses each line according to CSV_Columns configuration
   3. Each line is converted into a Bar object
   4. Column mapping is performed based on configuration
4. For API data:
   1. APIDataSource connects to the specified API endpoint
   2. API requests are made to fetch historical data
   3. Response data is parsed and converted to Bar objects
5. Bars are collected into a vector and sorted by timestamp
6. Optional partial data loading is performed if USE_PARTIAL_DATA is true
7. The loaded data is stored in the BacktestEngine's historicalData vector

4. Strategy Setup

After data loading, a strategy is created and set:

// In main.cpp
std::string stratType = config.getNested("/Strategy/Type", "Random");
std::unique_ptr strategy;

if (stratType == "ML") {
    strategy = std::make_unique();
} else if (stratType == "Benchmark") {
    strategy = std::make_unique();
} else {
    strategy = std::make_unique();
}

Utils::logMessage("Main: Creating " + strategy->getName() + " strategy.");
engine->setStrategy(std::move(strategy));

Strategy setup involves:

1. Creating an instance of the appropriate strategy type based on configuration
2. Setting strategy ownership to the BacktestEngine via setStrategy()
3. BacktestEngine provides the strategy with references to:
   • The Broker instance: strategy->setBroker(broker.get())
   • The historical data: strategy->setData(&historicalData, primaryDataName)
   • The configuration: strategy->setConfig(&config)
4. Strategy's initialize() method is called to set up internal state

5. Backtest Execution

The backtest is executed with:

// In main.cpp
engine->run();

The BacktestEngine::run() method performs the main backtest loop:

// In BacktestEngine::run()
// Call strategy's init method before starting
strategy->init();

// Loop through each bar in the historical data
for (size_t i = 0; i < historicalData.size(); ++i) {
    currentBarIndex = i;
    const Bar& currentBar = historicalData[i];
    
    // Update current price
    currentPrice = currentBar.columns[1]; // Assuming close price is at index 1
    
    // Process pending orders using current bar data
    broker->processOrders(currentBar);
    
    // Call strategy's next method to process this bar
    strategy->next(currentBar, i, currentPrice);
}

// Call strategy's stop method when done
strategy->stop();

For each bar in the historical data:

1. Current price information is updated
2. The Broker processes any pending orders:
   1. Orders are filled based on the current bar's price data
   2. Take profit and stop loss orders are checked
   3. Order statuses are updated
   4. The Strategy is notified of order status changes
3. The Strategy's next() method is called with the current bar:
   1. Strategy analyzes the bar using its internal logic
   2. It may submit new orders to the Broker
   3. It updates internal indicators and state

6. Strategy-Broker Interaction

The Strategy and Broker interact during the backtest loop:

Order Submission

// In a Strategy implementation
Order buyOrder;
buyOrder.type = OrderType::BUY;
buyOrder.symbol = bar.symbol;
buyOrder.requestedSize = 1.0;
buyOrder.requestedPrice = bar.columns[1]; // Close price
buyOrder.takeProfit = bar.columns[1] + takeProfitPips_;
buyOrder.stopLoss = bar.columns[1] - stopLossPips_;

// Submit to broker
int orderId = broker->submitOrder(buyOrder);

Order Processing

// In Broker::processOrders
// For each pending order
for (auto& order : pendingOrders) {
    // Check if the order can be executed
    if (canExecuteOrder(order, currentBar)) {
        // If it's an opening order
        if (!hasPosition(order.symbol) || 
            (hasPosition(order.symbol) && isOrderReducingPosition(order))) {
            executeOpenOrder(order, currentBar);
        }
        // If it's a closing order targeting an existing position
        else if (hasPosition(order.symbol)) {
            auto& position = positions[order.symbol];
            executeCloseOrder(order, position, currentBar);
        }
    }
    else {
        // Reject the order if it can't be executed
        rejectOrder(order, OrderStatus::REJECTED, currentBar);
    }
    
    // Notify the strategy of order status change
    if (strategy) {
        strategy->notifyOrder(order);
    }
}

Position Management

The Broker maintains and updates positions:

• Opening positions when orders are filled
• Updating positions when additional orders are filled
• Closing positions when opposing orders are filled
• Processing stop-loss and take-profit levels
• Calculating profit/loss and updating cash balances

7. Machine Learning Strategy Flow

For ML strategies like HMMStrategy, there's an additional flow for model interaction:

1. Loading ONNX models via OnnxModelInterface
2. Feature extraction from bar data
3. State detection using HMM model
4. Model inference to generate trading signals
5. Order creation based on model outputs

// In HMMStrategy::next
// Extract features from historical data
// ...

// Detect regime using HMM model
int regime = detectRegime(features);

// Select appropriate model for current regime
auto& model = regimeModels_[regime];

// Run inference
std::vector prediction = model->Predict(inputData, inputShape);

// Generate trading signal based on prediction
double signal = prediction[0];

// Create orders based on signal
if (signal > entryThreshold_) {
    // Create buy order
    // ...
} else if (signal < -entryThreshold_) {
    // Create sell order
    // ...
}

8. Metrics Calculation

After the backtest completes, performance metrics are calculated:

// At the end of BacktestEngine::run
// Calculate final portfolio value
double finalValue = broker->getValue(currentPrice);

// Calculate ROI
double roi = (finalValue - broker->getStartingCash()) / broker->getStartingCash() * 100.0;

// Log results
Utils::logMessage("Backtest completed.");
Utils::logMessage("Starting balance: $" + std::to_string(broker->getStartingCash()));
Utils::logMessage("Final balance: $" + std::to_string(finalValue));
Utils::logMessage("ROI: " + std::to_string(roi) + "%");

More detailed metrics may include:

• Sharpe ratio and sortino ratio
• Maximum drawdown percentage and duration
• Win/loss ratio
• Average profit per trade
• Total number of trades

9. Python Integration Flow

When using CppBacktester from Python via pybind11, the flow is similar:

# Python example
import cppbacktester_py as cb

# 1. Load configuration
config = cb.Config()
config.load_from_file("config.json")

# 2. Create engine with config
engine = cb.BacktestEngine(config)

# 3. Load data
engine.load_data()

# 4. Create and set strategy
strategy = cb.RandomStrategy()  # or HMMStrategy, BenchmarkStrategy, etc.
engine.set_strategy(strategy)

# 5. Run backtest
engine.run()

# 6. Get results
# (Future functionality)

The Python bindings expose the core C++ functionality, allowing:

• Configuration management
• Data loading
• Strategy selection and configuration
• Backtest execution
• Results analysis (in development)

Note: Some advanced features like custom Python strategies, direct DataFrame integration, and visualization tools are under development.