Evolutionary Machine Learning Stock Optimizer

Project Overview #

I prefer swing trading ordinary shares, but I simply don’t have the time to sit and stare at charts all day. To solve this, I built an automated trading system. By mixing Reinforcement Learning (RL) and evolutionary algorithms, the engine constantly trains and tests virtual trading agents on historical market data. The main goal is optimizing the Sharpe Ratio to maximize returns while managing risk so the system can make decisions on its own.

Evolve Trader UI and Sharpe Ratio Chart — Equity curve after 100 training cycles. Mostly follows S&P 500 performance when limited training is applied, due to true randomness being roughly equal to simply holding.

System Architecture #

The project ties together an automated data pipeline, parallel reinforcement learning environments, and a live web dashboard to monitor everything in real time.

Backend: Powered by FastAPI and Uvicorn, which acts as the central brain to handle background training requests and serve the UI.
Data Pipeline: I hooked the system up to the yfinance API to pull historical prices and automatically compile technical indicators like SMA, RSI, and MACD using NumPy and Pandas. It caches everything locally so I don’t hit API limits during heavy generational training runs.
The AI Stack: Built on Stable Baselines3 (using PPO and A2C algorithms) running on top of custom Gymnasium trading environments.
Frontend Dashboard: A clean HTML/JS interface that polls the backend to show live training metrics and Sharpe ratios on an interactive chart.

The Custom Gymnasium Environment #

The actual trading logic lives inside a custom SimpleTradingEnv class that plugs right into Stable Baselines3. It basically simulates a market for a single stock where the AI learns how to manage a portfolio’s cash and equity.

Action and Observation Space #

What does the AI actually see? It looks at a continuous observation vector containing the current Close price, SMA, RSI, MACD, how much stock we currently hold, and our available cash. Based on that snapshot, it outputs a decision between -1.0 and 1.0, scaling anywhere from going 100% short to 100% long on the position.

Reward Function #

To train the AI, it needs to know if it’s making good trades. The environment grades the agent step-by-step, calculating its reward based on the daily change in total equity (cash plus the actual value of the holdings). I also built in simulated slippage and trading fees to keep the training as realistic as possible.

#Calculating the reward via equity delta
equity_before = self._mark_to_market(price)
target_position_value = float(np.clip(a, -self.max_w, self.max_w)) * equity_before
current_position_value = float(self.position * price)
delta_value = target_position_value - current_position_value

# trade execution and fee deduction ...

equity_after = self._mark_to_market(new_price)
reward = float(equity_after - equity_before)