Updated file directory structure, added old PPO.

src/Machine-Learning/LSTM-python/src/.gitignore

@@ -1 +0,0 @@
-venv/

src/Machine-Learning/LSTM-python/src/.python-version

@@ -1 +0,0 @@
-3.9.13

src/Machine-Learning/LSTM-python/src/requirements.txt

@@ -1,13 +0,0 @@
-contourpy==1.3.0
-cycler==0.12.1
-fonttools==4.56.0
-importlib_resources==6.5.2
-kiwisolver==1.4.7
-matplotlib==3.9.4
-numpy==2.0.2
-packaging==24.2
-pillow==11.1.0
-pyparsing==3.2.1
-python-dateutil==2.9.0.post0
-six==1.17.0
-zipp==3.21.0

src/Machine-Learning/README.md

@@ -1 +0,0 @@
-THis is a simple feed forward MLP and I want to make it a LSTM/GRU in pure C baby.

src/MidasAgent/DQN/output/FuturesPPO.log

@@ -0,0 +1,48 @@
+2025-03-26 03:01:52,506 - INFO - ===== Resource Statistics =====
+2025-03-26 03:01:52,506 - INFO - Physical CPU Cores: 28
+2025-03-26 03:01:52,506 - INFO - Logical CPU Cores: 56
+2025-03-26 03:01:52,507 - INFO - CPU Usage per Core: [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 5.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]%
+2025-03-26 03:01:52,507 - INFO - No GPUs detected.
+2025-03-26 03:01:52,507 - INFO - =================================
+2025-03-26 03:01:52,507 - INFO - Configured TensorFlow to use CPU with optimized thread settings.
+2025-03-26 03:01:52,508 - INFO - Loading data from: data/MES2023Z.csv
+2025-03-26 03:01:52,513 - ERROR - Unexpected error: Missing column provided to 'parse_dates': 'time'
+2025-03-26 03:04:50,616 - INFO - ===== Resource Statistics =====
+2025-03-26 03:04:50,616 - INFO - Physical CPU Cores: 28
+2025-03-26 03:04:50,616 - INFO - Logical CPU Cores: 56
+2025-03-26 03:04:50,616 - INFO - CPU Usage per Core: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0]%
+2025-03-26 03:04:50,617 - INFO - No GPUs detected.
+2025-03-26 03:04:50,617 - INFO - =================================
+2025-03-26 03:04:50,617 - INFO - Configured TensorFlow to use CPU with optimized thread settings.
+2025-03-26 03:04:50,618 - INFO - Loading data from: data/MES2023Z.csv
+2025-03-26 03:04:50,621 - ERROR - Unexpected error: Missing column provided to 'parse_dates': 'time'
+2025-03-26 03:08:02,316 - INFO - ===== Resource Statistics =====
+2025-03-26 03:08:02,316 - INFO - Physical CPU Cores: 28
+2025-03-26 03:08:02,316 - INFO - Logical CPU Cores: 56
+2025-03-26 03:08:02,317 - INFO - CPU Usage per Core: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]%
+2025-03-26 03:08:02,317 - INFO - No GPUs detected.
+2025-03-26 03:08:02,317 - INFO - =================================
+2025-03-26 03:08:02,317 - INFO - Configured TensorFlow to use CPU with optimized thread settings.
+2025-03-26 03:08:02,318 - INFO - Loading data from: data/MES2023Z.csv
+2025-03-26 03:08:02,355 - INFO - Data columns after renaming: ['Date', 'Open', 'High', 'Low', 'Close', 'Volume']
+2025-03-26 03:08:02,383 - INFO - Data loaded and sorted successfully.
+2025-03-26 03:08:02,383 - INFO - Calculating technical indicators...
+2025-03-26 03:08:02,448 - INFO - Technical indicators calculated successfully.
+2025-03-26 03:08:02,464 - INFO - Starting parallel feature engineering with 54 workers...
+2025-03-26 03:08:03,331 - INFO - Parallel feature engineering completed.
+2025-03-26 03:08:03,341 - INFO - Training sequences shape: (676, 15, 17)
+2025-03-26 03:08:03,342 - INFO - Validation sequences shape: (144, 15, 17)
+2025-03-26 03:08:03,342 - INFO - Testing sequences shape: (146, 15, 17)
+2025-03-26 03:08:03,342 - INFO - Starting LSTM hyperparameter optimization with Optuna using 54 parallel trials...
+2025-03-26 03:22:04,033 - INFO - Best LSTM Hyperparameters: {'num_lstm_layers': 2, 'lstm_units': 64, 'dropout_rate': 0.13619292923712067, 'learning_rate': 0.0030545284525912166, 'optimizer': 'Nadam', 'decay': 9.615099767236892e-05}
+2025-03-26 03:22:04,553 - INFO - Training best LSTM model with optimized hyperparameters...
+2025-03-26 03:24:28,296 - INFO - Evaluating final LSTM model...
+2025-03-26 03:24:29,722 - INFO - Test MSE: 0.3437
+2025-03-26 03:24:29,722 - INFO - Test RMSE: 0.5862
+2025-03-26 03:24:29,722 - INFO - Test MAE: 0.4561
+2025-03-26 03:24:29,722 - INFO - Test R2 Score: 0.8620
+2025-03-26 03:24:29,722 - INFO - Directional Accuracy: 0.2759
+2025-03-26 03:24:30,013 - WARNING - You are saving your model as an HDF5 file via `model.save()` or `keras.saving.save_model(model)`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')` or `keras.saving.save_model(model, 'my_model.keras')`. 
+2025-03-26 03:24:30,121 - INFO - Saved best LSTM model and scaler objects.
+2025-03-26 03:24:30,150 - INFO - Starting PPO training...
+2025-03-26 05:47:15,571 - INFO - PPO training completed and model saved.

src/MidasAgent/DQN/output/LSTMDQN.log

@@ -104,3 +104,14 @@
 2025-03-06 20:54:21,265 - INFO - Scaled validation target shape: (3028,)
 2025-03-06 20:54:21,265 - INFO - Scaled testing target shape: (3030,)
 2025-03-06 20:54:21,265 - INFO - Starting LSTM hyperparameter optimization with Optuna using 54 parallel trials...
+2025-03-06 23:10:28,345 - INFO - Best LSTM Hyperparameters: {'num_lstm_layers': 2, 'lstm_units': 96, 'dropout_rate': 0.18300207247480796, 'learning_rate': 0.0015024264996830019, 'optimizer': 'Nadam', 'decay': 6.153016040618131e-07}
+2025-03-06 23:10:28,788 - INFO - Training best LSTM model with optimized hyperparameters...
+2025-03-07 01:26:27,312 - INFO - Evaluating final LSTM model...
+2025-03-07 01:26:29,772 - INFO - Test MSE: 0.0922
+2025-03-07 01:26:29,773 - INFO - Test RMSE: 0.3037
+2025-03-07 01:26:29,773 - INFO - Test MAE: 0.1935
+2025-03-07 01:26:29,773 - INFO - Test R2 Score: 0.9924
+2025-03-07 01:26:29,773 - INFO - Directional Accuracy: 0.4780
+2025-03-07 01:26:30,075 - WARNING - You are saving your model as an HDF5 file via `model.save()` or `keras.saving.save_model(model)`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')` or `keras.saving.save_model(model, 'my_model.keras')`. 
+2025-03-07 01:26:30,135 - INFO - Saved best LSTM model and scaler objects (best_lstm_model.h5, scaler_features.pkl, scaler_target.pkl).
+2025-03-07 01:26:30,135 - INFO - Training DQN agent: Attempt 1 with hyperparameters: {'lr': 0.001, 'gamma': 0.95, 'exploration_fraction': 0.1, 'buffer_size': 10000, 'batch_size': 64}

src/MidasAgent/PPO/LSTMPPO.py

@@ -0,0 +1,754 @@
+import os
+import sys
+import argparse
+import numpy as np
+import pandas as pd
+import logging
+from tabulate import tabulate
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+import psutil
+import GPUtil
+import tensorflow as tf
+from tensorflow.keras.models import Sequential, load_model
+from tensorflow.keras.layers import LSTM, Dense, Dropout, Bidirectional
+from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau
+from tensorflow.keras.losses import Huber
+from tensorflow.keras.regularizers import l2
+from tensorflow.keras.optimizers import Adam, Nadam
+
+from sklearn.preprocessing import MinMaxScaler
+from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
+import joblib
+import optuna
+from optuna.integration import KerasPruningCallback
+
+import gym
+from gym import spaces
+from stable_baselines3 import PPO
+from stable_baselines3.common.vec_env import DummyVecEnv
+
+from multiprocessing import Pool, cpu_count
+import threading
+import time
+
+# Suppress TensorFlow logs beyond errors
+os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
+
+# =============================================================================
+# Resource Detection Functions
+# =============================================================================
+def get_cpu_info():
+    cpu_count_physical = psutil.cpu_count(logical=False)  # Physical cores
+    cpu_count_logical = psutil.cpu_count(logical=True)      # Logical cores
+    cpu_percent = psutil.cpu_percent(interval=1, percpu=True)
+    return {
+        'physical_cores': cpu_count_physical,
+        'logical_cores': cpu_count_logical,
+        'cpu_percent': cpu_percent
+    }
+
+def get_gpu_info():
+    gpus = GPUtil.getGPUs()
+    gpu_info = []
+    for gpu in gpus:
+        gpu_info.append({
+            'id': gpu.id,
+            'name': gpu.name,
+            'load': gpu.load * 100,  # Convert to percentage
+            'memory_total': gpu.memoryTotal,
+            'memory_used': gpu.memoryUsed,
+            'memory_free': gpu.memoryFree,
+            'temperature': gpu.temperature
+        })
+    return gpu_info
+
+def configure_tensorflow(cpu_stats, gpu_stats):
+    logical_cores = cpu_stats['logical_cores']
+    os.environ["OMP_NUM_THREADS"] = str(logical_cores)
+    os.environ["TF_NUM_INTRAOP_THREADS"] = str(logical_cores)
+    os.environ["TF_NUM_INTEROP_THREADS"] = str(logical_cores)
+    
+    if gpu_stats:
+        gpus = tf.config.list_physical_devices('GPU')
+        if gpus:
+            try:
+                for gpu in gpus:
+                    tf.config.experimental.set_memory_growth(gpu, True)
+                logging.info(f"Enabled memory growth for {len(gpus)} GPU(s).")
+            except RuntimeError as e:
+                logging.error(f"TensorFlow GPU configuration error: {e}")
+    else:
+        tf.config.threading.set_intra_op_parallelism_threads(logical_cores)
+        tf.config.threading.set_inter_op_parallelism_threads(logical_cores)
+        logging.info("Configured TensorFlow to use CPU with optimized thread settings.")
+
+def monitor_resources(interval=60):
+    while True:
+        cpu = psutil.cpu_percent(interval=1, percpu=True)
+        gpu = get_gpu_info()
+        logging.info(f"CPU Usage per Core: {cpu}%")
+        if gpu:
+            for gpu_stat in gpu:
+                logging.info(f"GPU {gpu_stat['id']} - {gpu_stat['name']}: Load: {gpu_stat['load']}%, "
+                             f"Memory Used: {gpu_stat['memory_used']}MB / {gpu_stat['memory_total']}MB, "
+                             f"Temperature: {gpu_stat['temperature']}°C")
+        else:
+            logging.info("No GPUs detected.")
+        logging.info("-" * 50)
+        time.sleep(interval)
+
+# =============================================================================
+# Data Loading & Technical Indicators
+# =============================================================================
+def load_data(file_path):
+    logging.info(f"Loading data from: {file_path}")
+    try:
+        df = pd.read_csv(file_path, parse_dates=['time'])
+    except FileNotFoundError:
+        logging.error(f"File not found: {file_path}")
+        sys.exit(1)
+    except pd.errors.ParserError as e:
+        logging.error(f"Error parsing CSV file: {e}")
+        sys.exit(1)
+    except Exception as e:
+        logging.error(f"Unexpected error: {e}")
+        sys.exit(1)
+
+    rename_mapping = {
+        'time': 'Date',
+        'open': 'Open',
+        'high': 'High',
+        'low': 'Low',
+        'close': 'Close'
+    }
+    df.rename(columns=rename_mapping, inplace=True)
+    logging.info(f"Data columns after renaming: {df.columns.tolist()}")
+    df.sort_values('Date', inplace=True)
+    df.reset_index(drop=True, inplace=True)
+    logging.info("Data loaded and sorted successfully.")
+    return df
+
+def compute_rsi(series, window=14):
+    delta = series.diff()
+    gain = delta.where(delta > 0, 0).rolling(window=window).mean()
+    loss = -delta.where(delta < 0, 0).rolling(window=window).mean()
+    RS = gain / (loss + 1e-9)
+    return 100 - (100 / (1 + RS))
+
+def compute_macd(series, span_short=12, span_long=26, span_signal=9):
+    ema_short = series.ewm(span=span_short, adjust=False).mean()
+    ema_long  = series.ewm(span=span_long, adjust=False).mean()
+    macd_line = ema_short - ema_long
+    signal_line = macd_line.ewm(span=span_signal, adjust=False).mean()
+    return macd_line - signal_line  # histogram
+
+def compute_obv(df):
+    signed_volume = (np.sign(df['Close'].diff()) * df['Volume']).fillna(0)
+    return signed_volume.cumsum()
+
+def compute_adx(df, window=14):
+    df['H-L']  = df['High'] - df['Low']
+    df['H-Cp'] = (df['High'] - df['Close'].shift(1)).abs()
+    df['L-Cp'] = (df['Low']  - df['Close'].shift(1)).abs()
+    tr = df[['H-L','H-Cp','L-Cp']].max(axis=1)
+    tr_rolling = tr.rolling(window=window).mean()
+    adx_placeholder = tr_rolling / (df['Close'] + 1e-9)
+    df.drop(['H-L','H-Cp','L-Cp'], axis=1, inplace=True)
+    return adx_placeholder
+
+def compute_bollinger_bands(series, window=20, num_std=2):
+    sma = series.rolling(window=window).mean()
+    std = series.rolling(window=window).std()
+    upper = sma + num_std * std
+    lower = sma - num_std * std
+    bandwidth = (upper - lower) / (sma + 1e-9)
+    return upper, lower, bandwidth
+
+def compute_mfi(df, window=14):
+    typical_price = (df['High'] + df['Low'] + df['Close']) / 3
+    money_flow = typical_price * df['Volume']
+    prev_tp = typical_price.shift(1)
+    flow_pos = money_flow.where(typical_price > prev_tp, 0)
+    flow_neg = money_flow.where(typical_price < prev_tp, 0)
+    pos_sum = flow_pos.rolling(window=window).sum()
+    neg_sum = flow_neg.rolling(window=window).sum()
+    mfi = 100 - (100 / (1 + pos_sum / (neg_sum + 1e-9)))
+    return mfi
+
+def calculate_technical_indicators(df):
+    logging.info("Calculating technical indicators...")
+    df['RSI'] = compute_rsi(df['Close'], 14)
+    df['MACD'] = compute_macd(df['Close'])
+    df['OBV'] = compute_obv(df)
+    df['ADX'] = compute_adx(df)
+
+    up, lo, bw = compute_bollinger_bands(df['Close'], 20, 2)
+    df['BB_Upper'] = up
+    df['BB_Lower'] = lo
+    df['BB_Width'] = bw
+
+    df['MFI'] = compute_mfi(df, 14)
+    df['SMA_5'] = df['Close'].rolling(5).mean()
+    df['SMA_10'] = df['Close'].rolling(10).mean()
+    df['EMA_5'] = df['Close'].ewm(span=5, adjust=False).mean()
+    df['EMA_10'] = df['Close'].ewm(span=10, adjust=False).mean()
+    df['STDDEV_5'] = df['Close'].rolling(5).std()
+
+    df.dropna(inplace=True)
+    logging.info("Technical indicators calculated successfully.")
+    return df
+
+# =============================================================================
+# Argument Parsing
+# =============================================================================
+def parse_arguments():
+    parser = argparse.ArgumentParser(description='Futures Trading with LSTM Forecasting and PPO.')
+    parser.add_argument('csv_path', type=str,
+                        help='Path to CSV data with columns [time, open, high, low, close, volume].')
+    parser.add_argument('--lstm_window_size', type=int, default=15,
+                        help='Sequence window size for LSTM forecasting. Default=15.')
+    parser.add_argument('--ppo_total_timesteps', type=int, default=100000,
+                        help='Total timesteps to train the PPO model. Default=100000.')
+    parser.add_argument('--n_trials_lstm', type=int, default=30,
+                        help='Number of Optuna trials for LSTM hyperparameter tuning. Default=30.')
+    parser.add_argument('--preprocess_workers', type=int, default=None,
+                        help='Number of worker processes for data preprocessing. Defaults to (logical cores - 2).')
+    parser.add_argument('--monitor_resources', action='store_true',
+                        help='Enable real-time resource monitoring.')
+    parser.add_argument('--output_dir', type=str, default='output',
+                        help='Directory where all output files will be saved.')
+    parser.add_argument('--action_mode', type=str, choices=['discrete', 'continuous'], default='discrete',
+                        help='Select action space type: discrete (e.g., -5 to +5) or continuous (Box). Default=discrete.')
+    parser.add_argument('--max_contracts', type=int, default=5,
+                        help='Maximum number of contracts to trade per action. Default=5.')
+    return parser.parse_args()
+
+# =============================================================================
+# LSTM Price Predictor (renamed from LSTM part)
+# =============================================================================
+def build_lstm(input_shape, hyperparams):
+    model = Sequential()
+    num_layers = hyperparams['num_lstm_layers']
+    units      = hyperparams['lstm_units']
+    drop       = hyperparams['dropout_rate']
+    for i in range(num_layers):
+        return_seqs = (i < num_layers - 1)
+        if i == 0:
+            model.add(Bidirectional(LSTM(units, return_sequences=return_seqs, kernel_regularizer=l2(1e-4)),
+                                    input_shape=input_shape))
+        else:
+            model.add(Bidirectional(LSTM(units, return_sequences=return_seqs, kernel_regularizer=l2(1e-4))))
+        model.add(Dropout(drop))
+    model.add(Dense(1, activation='linear'))
+
+    opt_name = hyperparams['optimizer']
+    lr       = hyperparams['learning_rate']
+    decay    = hyperparams['decay']
+    if opt_name == 'Adam':
+        opt = Adam(learning_rate=lr, decay=decay)
+    elif opt_name == 'Nadam':
+        opt = Nadam(learning_rate=lr)
+    else:
+        opt = Adam(learning_rate=lr)
+
+    model.compile(loss=Huber(), optimizer=opt, metrics=['mae'])
+    return model
+
+# =============================================================================
+# Custom Gym Environment for Futures Trading with LSTM Forecasting
+# =============================================================================
+class FuturesTradingEnv(gym.Env):
+    """
+    A custom OpenAI Gym environment for futures trading.
+    It integrates an LSTM price predictor (PricePredictorLSTM) for forecasting.
+    The environment tracks positions as contracts_held (can be negative for shorts).
+    Reward is defined as the change in mark-to-market profit (unrealized PnL)
+    minus transaction costs.
+    The action space can be either discrete (e.g., -max_contracts ... +max_contracts)
+    or continuous (Box space) which is then rounded to an integer.
+    """
+    metadata = {'render.modes': ['human']}
+
+    def __init__(self, df, feature_columns, lstm_model, scaler_features, scaler_target,
+                 window_size=15, transaction_cost=0.001, action_mode='discrete', max_contracts=5):
+        super(FuturesTradingEnv, self).__init__()
+        self.df = df.reset_index(drop=True)
+        self.feature_columns = feature_columns
+        self.lstm_model = lstm_model  # PricePredictorLSTM (frozen during training)
+        self.scaler_features = scaler_features
+        self.scaler_target = scaler_target
+        self.window_size = window_size
+        self.transaction_cost = transaction_cost
+        self.action_mode = action_mode
+        self.max_contracts = max_contracts
+
+        self.max_steps = len(df)
+        self.current_step = 0
+
+        # Futures position variables
+        self.contracts_held = 0  # positive for long, negative for short
+        self.entry_price = None  # weighted average entry price
+
+        # Pre-calculate normalized features for observations
+        self.raw_features = df[feature_columns].values
+
+        # Define action space
+        if self.action_mode == 'discrete':
+            # Actions: integer orders from -max_contracts to +max_contracts
+            self.action_space = spaces.Discrete(2 * self.max_contracts + 1)
+        else:
+            # Continuous action: a real number in [-max_contracts, max_contracts]
+            self.action_space = spaces.Box(low=-self.max_contracts, high=self.max_contracts, shape=(1,), dtype=np.float32)
+
+        # Observation space: technical indicators + [normalized contracts_held, normalized unrealized PnL] + LSTM forecast
+        obs_len = len(feature_columns) + 2 + 1
+        self.observation_space = spaces.Box(low=-np.inf, high=np.inf, shape=(obs_len,), dtype=np.float32)
+
+        # Lock for LSTM prediction (if used in multi-threaded settings)
+        self.lstm_lock = threading.Lock()
+
+    def reset(self):
+        self.current_step = 0
+        self.contracts_held = 0
+        self.entry_price = None
+        return self._get_obs()
+
+    def _get_obs(self):
+        # Normalize the raw features row by row
+        row = self.raw_features[self.current_step]
+        row_max = np.max(np.abs(row)) if np.max(np.abs(row)) != 0 else 1.0
+        row_norm = row / row_max
+
+        # Additional account info:
+        # Normalize contracts_held by max_contracts.
+        # Unrealized PnL: if entry_price exists, (current_price - entry_price)*contracts_held; else 0.
+        current_price = self.df.loc[self.current_step, 'Close']
+        pnl = (current_price - self.entry_price) * self.contracts_held if self.entry_price is not None else 0.0
+
+        additional = np.array([
+            self.contracts_held / self.max_contracts,
+            pnl  # You might choose to normalize pnl as needed
+        ], dtype=np.float32)
+
+        # LSTM price forecast: predict next price if possible
+        if self.current_step < self.window_size:
+            forecast = 0.0
+        else:
+            seq = self.raw_features[self.current_step - self.window_size: self.current_step]
+            seq_scaled = self.scaler_features.transform(seq)
+            seq_scaled = np.expand_dims(seq_scaled, axis=0)  # shape: (1, window_size, num_features)
+            with self.lstm_lock:
+                pred_scaled = self.lstm_model.predict(seq_scaled, verbose=0).flatten()[0]
+            pred_scaled = np.clip(pred_scaled, 0, 1)
+            unscaled = self.scaler_target.inverse_transform([[pred_scaled]])[0, 0]
+            # Forecast can be represented as the relative difference from the current price
+            forecast = (unscaled - current_price) / (current_price + 1e-9)
+
+        obs = np.concatenate([row_norm, additional, [forecast]]).astype(np.float32)
+        return obs
+
+    def step(self, action):
+        prev_price = self.df.loc[self.current_step, 'Close']
+        prev_position = self.contracts_held
+
+        # Convert action to an integer number of contracts
+        if self.action_mode == 'discrete':
+            # Discrete action space: 0 corresponds to -max_contracts, last to +max_contracts.
+            action_int = action - self.max_contracts
+        else:
+            # For continuous, round to nearest integer
+            action_int = int(np.round(action[0]))
+            # Clip action_int to allowable range
+            action_int = np.clip(action_int, -self.max_contracts, self.max_contracts)
+
+        # Transaction cost fee applied on the notional of new trade
+        current_price = self.df.loc[self.current_step, 'Close']
+        fee = self.transaction_cost * abs(action_int) * current_price
+
+        # Update position
+        if action_int != 0:
+            # If no current position, simply set the new position and record entry price
+            if self.contracts_held == 0:
+                self.contracts_held = action_int
+                self.entry_price = current_price
+            # If same sign, update weighted average entry price
+            elif np.sign(self.contracts_held) == np.sign(action_int):
+                total_contracts = self.contracts_held + action_int
+                self.entry_price = (self.entry_price * self.contracts_held + current_price * action_int) / total_contracts
+                self.contracts_held = total_contracts
+            # If opposite sign, reduce/flip position:
+            else:
+                # If the new action fully reverses the position, calculate remaining contracts
+                if abs(action_int) >= abs(self.contracts_held):
+                    # Realize profit/loss on the closed position (will be reflected in reward)
+                    self.contracts_held = self.contracts_held + action_int  # may flip sign
+                    self.entry_price = current_price if self.contracts_held != 0 else None
+                else:
+                    # Partial close; position sign remains the same.
+                    self.contracts_held = self.contracts_held + action_int
+                    # entry_price remains unchanged
+
+        # Mark-to-market PnL: change from previous price * previous position
+        pnl_change = (current_price - prev_price) * prev_position
+        # Reward: change in unrealized PnL minus transaction fees
+        reward = pnl_change - fee
+
+        self.current_step += 1
+        done = (self.current_step >= self.max_steps - 1)
+        obs = self._get_obs()
+        return obs, reward, done, {}
+
+    def render(self, mode='human'):
+        current_price = self.df.loc[self.current_step, 'Close']
+        pnl = (current_price - self.entry_price) * self.contracts_held if self.entry_price is not None else 0.0
+        print(f"Step: {self.current_step}, Contracts Held: {self.contracts_held}, "
+              f"Entry Price: {self.entry_price}, Current Price: {current_price:.2f}, PnL: {pnl:.2f}")
+
+# =============================================================================
+# Placeholders for Live Deployment Functions
+# =============================================================================
+def get_live_data():
+    """
+    Placeholder: Connect to a live data feed and return the latest market data.
+    """
+    # Implement connection to a live data source (e.g., API call to a broker)
+    # Return a dictionary or DataFrame row with market data.
+    return None
+
+def execute_order(action):
+    """
+    Placeholder: Execute the trading order in a live environment.
+    """
+    # Implement order execution logic with your broker API.
+    logging.info(f"Executing order: {action}")
+
+def live_trading_loop(model, env, polling_interval=5):
+    """
+    Example live trading loop.
+    At each step, get live data, update the environment, use the model to predict the next action,
+    and execute the order.
+    """
+    obs = env.reset()
+    done = False
+    while not done:
+        live_data = get_live_data()
+        if live_data is not None:
+            # Update environment with live data (this requires proper integration)
+            # For example, you might append the live data to the internal dataframe.
+            pass
+
+        # Use the PPO model to predict the next action
+        action, _ = model.predict(obs, deterministic=True)
+        execute_order(action)
+        obs, reward, done, _ = env.step(action)
+        env.render()
+        time.sleep(polling_interval)
+
+# =============================================================================
+# Data Preprocessing with Parallelization
+# =============================================================================
+def parallel_feature_engineering(row):
+    """
+    Placeholder function for additional feature engineering.
+    """
+    return row
+
+def feature_engineering_parallel(df, num_workers):
+    logging.info(f"Starting parallel feature engineering with {num_workers} workers...")
+    with Pool(processes=num_workers) as pool:
+        processed_rows = pool.map(parallel_feature_engineering, [row for _, row in df.iterrows()])
+    df_processed = pd.DataFrame(processed_rows)
+    logging.info("Parallel feature engineering completed.")
+    return df_processed
+
+# =============================================================================
+# MAIN FUNCTION: LSTM Training + PPO for Futures Trading
+# =============================================================================
+def main():
+    args = parse_arguments()
+    csv_path = args.csv_path
+    output_dir = args.output_dir
+    os.makedirs(output_dir, exist_ok=True)
+    lstm_window_size = args.lstm_window_size
+    ppo_total_timesteps = args.ppo_total_timesteps
+    n_trials_lstm = args.n_trials_lstm
+    preprocess_workers = args.preprocess_workers
+    enable_resource_monitor = args.monitor_resources
+    action_mode = args.action_mode
+    max_contracts = args.max_contracts
+
+    # -----------------------------
+    # Setup Logging
+    # -----------------------------
+    logging.basicConfig(level=logging.INFO,
+                        format='%(asctime)s - %(levelname)s - %(message)s',
+                        handlers=[
+                            logging.FileHandler(os.path.join(output_dir, "FuturesPPO.log")),
+                            logging.StreamHandler(sys.stdout)
+                        ])
+
+    # -----------------------------
+    # Resource Detection & Logging
+    # -----------------------------
+    cpu_stats = get_cpu_info()
+    gpu_stats = get_gpu_info()
+    logging.info("===== Resource Statistics =====")
+    logging.info(f"Physical CPU Cores: {cpu_stats['physical_cores']}")
+    logging.info(f"Logical CPU Cores: {cpu_stats['logical_cores']}")
+    logging.info(f"CPU Usage per Core: {cpu_stats['cpu_percent']}%")
+    if gpu_stats:
+        for gpu in gpu_stats:
+            logging.info(f"GPU {gpu['id']} - {gpu['name']}: Load: {gpu['load']}%, Memory Used: {gpu['memory_used']}MB/{gpu['memory_total']}MB, Temperature: {gpu['temperature']}°C")
+    else:
+        logging.info("No GPUs detected.")
+    logging.info("=================================")
+
+    # -----------------------------
+    # Configure TensorFlow
+    # -----------------------------
+    configure_tensorflow(cpu_stats, gpu_stats)
+
+    # -----------------------------
+    # Start Resource Monitoring (Optional)
+    # -----------------------------
+    if enable_resource_monitor:
+        logging.info("Starting real-time resource monitoring...")
+        resource_monitor_thread = threading.Thread(target=monitor_resources, args=(60,), daemon=True)
+        resource_monitor_thread.start()
+
+    ##########################################
+    # A) LSTM PART: LOAD, PREPROCESS & TUNE
+    ##########################################
+    df = load_data(csv_path)
+    df = calculate_technical_indicators(df)
+
+    feature_columns = [
+        'SMA_5','SMA_10','EMA_5','EMA_10','STDDEV_5',
+        'RSI','MACD','ADX','OBV','Volume','Open','High','Low',
+        'BB_Upper','BB_Lower','BB_Width','MFI'
+    ]
+    target_column = 'Close'
+    df = df[['Date'] + feature_columns + [target_column]].dropna()
+
+    # 2) Controlled Parallel Data Preprocessing
+    if preprocess_workers is None:
+        preprocess_workers = max(1, cpu_stats['logical_cores'] - 2)
+    else:
+        preprocess_workers = min(preprocess_workers, cpu_stats['logical_cores'])
+    df = feature_engineering_parallel(df, num_workers=preprocess_workers)
+
+    scaler_features = MinMaxScaler()
+    scaler_target   = MinMaxScaler()
+
+    X_all = df[feature_columns].values
+    y_all = df[[target_column]].values
+
+    X_scaled = scaler_features.fit_transform(X_all)
+    y_scaled = scaler_target.fit_transform(y_all).flatten()
+
+    # 3) Create sequences for LSTM forecasting
+    def create_sequences(features, target, window_size):
+        X_seq, y_seq = [], []
+        for i in range(len(features) - window_size):
+            X_seq.append(features[i:i+window_size])
+            y_seq.append(target[i+window_size])
+        return np.array(X_seq), np.array(y_seq)
+
+    X, y = create_sequences(X_scaled, y_scaled, lstm_window_size)
+
+    # 4) Split into train/val/test
+    train_size = int(len(X) * 0.7)
+    val_size   = int(len(X) * 0.15)
+    test_size  = len(X) - train_size - val_size
+
+    X_train, y_train = X[:train_size], y[:train_size]
+    X_val,   y_val   = X[train_size: train_size + val_size], y[train_size: train_size + val_size]
+    X_test,  y_test  = X[train_size + val_size:], y[train_size + val_size:]
+
+    logging.info(f"Training sequences shape: {X_train.shape}")
+    logging.info(f"Validation sequences shape: {X_val.shape}")
+    logging.info(f"Testing sequences shape: {X_test.shape}")
+
+    # 5) Define LSTM objective for hyperparameter tuning using Optuna
+    def lstm_objective(trial):
+        num_lstm_layers = trial.suggest_int('num_lstm_layers', 1, 3)
+        lstm_units      = trial.suggest_categorical('lstm_units', [32, 64, 96, 128])
+        dropout_rate    = trial.suggest_float('dropout_rate', 0.1, 0.5)
+        learning_rate   = trial.suggest_float('learning_rate', 1e-5, 1e-2, log=True)
+        optimizer_name  = trial.suggest_categorical('optimizer', ['Adam', 'Nadam'])
+        decay           = trial.suggest_float('decay', 0.0, 1e-4)
+
+        hyperparams = {
+            'num_lstm_layers': num_lstm_layers,
+            'lstm_units': lstm_units,
+            'dropout_rate': dropout_rate,
+            'learning_rate': learning_rate,
+            'optimizer': optimizer_name,
+            'decay': decay
+        }
+
+        model_ = build_lstm((X_train.shape[1], X_train.shape[2]), hyperparams)
+        early_stop = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True)
+        lr_reduce  = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=5, min_lr=1e-6)
+        cb_prune   = KerasPruningCallback(trial, 'val_loss')
+
+        history = model_.fit(
+            X_train, y_train,
+            epochs=100,
+            batch_size=16,
+            validation_data=(X_val, y_val),
+            callbacks=[early_stop, lr_reduce, cb_prune],
+            verbose=0
+        )
+        val_mae = min(history.history['val_mae'])
+        return val_mae
+
+    logging.info(f"Starting LSTM hyperparameter optimization with Optuna using {cpu_stats['logical_cores']-2} parallel trials...")
+    study_lstm = optuna.create_study(direction='minimize')
+    study_lstm.optimize(lstm_objective, n_trials=n_trials_lstm, n_jobs=cpu_stats['logical_cores']-2)
+    best_lstm_params = study_lstm.best_params
+    logging.info(f"Best LSTM Hyperparameters: {best_lstm_params}")
+
+    # 6) Train final LSTM (PricePredictorLSTM) with best hyperparameters
+    final_lstm = build_lstm((X_train.shape[1], X_train.shape[2]), best_lstm_params)
+    early_stop_final = EarlyStopping(monitor='val_loss', patience=20, restore_best_weights=True)
+    lr_reduce_final  = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=5, min_lr=1e-6)
+    logging.info("Training best LSTM model with optimized hyperparameters...")
+    final_lstm.fit(
+        X_train, y_train,
+        epochs=300,
+        batch_size=16,
+        validation_data=(X_val, y_val),
+        callbacks=[early_stop_final, lr_reduce_final],
+        verbose=1
+    )
+
+    # 7) Evaluate final LSTM
+    def evaluate_final_lstm(model, X_test, y_test):
+        logging.info("Evaluating final LSTM model...")
+        y_pred_scaled = model.predict(X_test).flatten()
+        y_pred_scaled = np.clip(y_pred_scaled, 0, 1)
+        y_pred = scaler_target.inverse_transform(y_pred_scaled.reshape(-1, 1)).flatten()
+        y_test_actual = scaler_target.inverse_transform(y_test.reshape(-1, 1)).flatten()
+
+        mse_ = mean_squared_error(y_test_actual, y_pred)
+        rmse_ = np.sqrt(mse_)
+        mae_ = mean_absolute_error(y_test_actual, y_pred)
+        r2_  = r2_score(y_test_actual, y_pred)
+
+        direction_actual = np.sign(np.diff(y_test_actual))
+        direction_pred  = np.sign(np.diff(y_pred))
+        directional_accuracy = np.mean(direction_actual == direction_pred)
+
+        logging.info(f"Test MSE: {mse_:.4f}")
+        logging.info(f"Test RMSE: {rmse_:.4f}")
+        logging.info(f"Test MAE: {mae_:.4f}")
+        logging.info(f"Test R2 Score: {r2_:.4f}")
+        logging.info(f"Directional Accuracy: {directional_accuracy:.4f}")
+
+        plt.figure(figsize=(14, 7))
+        plt.plot(y_test_actual, label='Actual Price')
+        plt.plot(y_pred, label='Predicted Price')
+        plt.title('LSTM: Actual vs Predicted Closing Prices')
+        plt.legend()
+        plt.grid(True)
+        plt.savefig(os.path.join(output_dir, 'lstm_actual_vs_pred.png'))
+        plt.close()
+
+        table = []
+        limit = min(40, len(y_test_actual))
+        for i in range(limit):
+            table.append([i, round(y_test_actual[i], 2), round(y_pred[i], 2)])
+        headers = ["Index", "Actual Price", "Predicted Price"]
+        print("\nFirst 40 Actual vs. Predicted Prices:")
+        print(tabulate(table, headers=headers, tablefmt="pretty"))
+        return r2_, directional_accuracy
+
+    _r2, _diracc = evaluate_final_lstm(final_lstm, X_test, y_test)
+
+    # 8) Save final LSTM model and scalers
+    final_lstm.save(os.path.join(output_dir, 'best_lstm_model.h5'))
+    joblib.dump(scaler_features, os.path.join(output_dir, 'scaler_features.pkl'))
+    joblib.dump(scaler_target, os.path.join(output_dir, 'scaler_target.pkl'))
+    logging.info("Saved best LSTM model and scaler objects.")
+
+    ##########################################
+    # B) PPO PART: SET UP FUTURES TRADING ENVIRONMENT
+    ##########################################
+    env_params = {
+        'df': df,
+        'feature_columns': feature_columns,
+        'lstm_model': final_lstm,  # Frozen LSTM for forecasting
+        'scaler_features': scaler_features,
+        'scaler_target': scaler_target,
+        'window_size': lstm_window_size,
+        'transaction_cost': 0.001,
+        'action_mode': action_mode,
+        'max_contracts': max_contracts
+    }
+
+    # Create the FuturesTradingEnv and wrap it for PPO training
+    env = FuturesTradingEnv(**env_params)
+    vec_env = DummyVecEnv([lambda: env])
+
+    # PPO hyperparameters (customize as needed)
+    ppo_hyperparams = {
+        'n_steps': 2048,
+        'batch_size': 64,
+        'gae_lambda': 0.95,
+        'gamma': 0.99,
+        'learning_rate': 3e-4,
+        'ent_coef': 0.0,
+        'verbose': 1
+    }
+
+    # -----------------------------
+    # Train PPO Model
+    # -----------------------------
+    logging.info("Starting PPO training...")
+    ppo_model = PPO('MlpPolicy', vec_env, **ppo_hyperparams)
+    ppo_model.learn(total_timesteps=ppo_total_timesteps)
+    ppo_model.save(os.path.join(output_dir, "best_ppo_model.zip"))
+    logging.info("PPO training completed and model saved.")
+
+    ##########################################
+    # C) FINAL INFERENCE & (Optional) LIVE TRADING EXAMPLE
+    ##########################################
+    # Evaluate the trained PPO model in the environment
+    obs = env.reset()
+    done = False
+    total_reward = 0.0
+    step_data = []
+    step_count = 0
+
+    while not done:
+        step_count += 1
+        action, _ = ppo_model.predict(obs, deterministic=True)
+        obs, reward, done, _ = env.step(action)
+        total_reward += reward
+        step_data.append({
+            "Step": step_count,
+            "Action": int(action) if action_mode=='discrete' else int(np.round(action[0])),
+            "Reward": reward,
+            "Contracts": env.contracts_held
+        })
+
+    final_pnl = (env.df.loc[env.current_step, 'Close'] - (env.entry_price if env.entry_price is not None else 0)) * env.contracts_held
+    print("\n=== Final PPO Inference ===")
+    print(f"Total Steps: {step_count}")
+    print(f"Final Contracts Held: {env.contracts_held}")
+    print(f"Final Estimated PnL: {final_pnl:.2f}")
+    print(f"Total Reward Sum: {total_reward:.2f}")
+
+    print("\nLast 15 Steps:")
+    last_n = step_data[-15:] if len(step_data) > 15 else step_data
+    print(tabulate(last_n, headers="keys", tablefmt="pretty"))
+
+    # OPTIONAL: Uncomment to run a live trading loop (requires implementation of live data feed and order execution)
+    # live_trading_loop(ppo_model, env)
+
+if __name__ == "__main__":
+    main()
+

src/MidasAgent/data/converter.py

@@ -0,0 +1,28 @@
+import csv
+
+# Input and output file names
+input_file = "MES2023Z.txt"  # Change this to your actual file name
+output_file = "MES2023Z.csv"
+
+# Open the TXT file and write to a CSV file
+with open(input_file, "r") as txt_file, open(output_file, "w", newline="") as csv_file:
+    writer = csv.writer(csv_file)
+    
+    # Write header row
+    writer.writerow(["Time", "Open", "High", "Low", "Close", "Volume"])
+    
+    for line in txt_file:
+        parts = line.strip().split(",")  # Split by comma
+        if len(parts) == 9:  # Ensure there are enough columns
+            time = parts[2]  # Extract time
+            open_price = parts[3]
+            high = parts[4]
+            low = parts[5]
+            close = parts[6]
+            volume = parts[7]
+
+            # Write row without the first two unnecessary columns
+            writer.writerow([time, open_price, high, low, close, volume])
+
+print(f"CSV file saved as {output_file}")
+

src/MidasAgent/requirements.txt

@@ -0,0 +1,95 @@
+absl-py==2.1.0
+alembic==1.15.1
+astunparse==1.6.3
+certifi==2025.1.31
+charset-normalizer==3.4.1
+cloudpickle==3.1.1
+colorlog==6.9.0
+contourpy==1.3.0
+cycler==0.12.1
+Farama-Notifications==0.0.4
+filelock==3.17.0
+flatbuffers==25.2.10
+fonttools==4.56.0
+fsspec==2025.2.0
+gast==0.6.0
+google-pasta==0.2.0
+GPUtil==1.4.0
+greenlet==3.1.1
+grpcio==1.70.0
+gym==0.26.2
+gym-notices==0.0.8
+gymnasium==1.0.0
+h5py==3.13.0
+idna==3.10
+importlib_metadata==8.6.1
+importlib_resources==6.5.2
+Jinja2==3.1.6
+joblib==1.4.2
+keras==3.9.0
+kiwisolver==1.4.7
+libclang==18.1.1
+Mako==1.3.9
+Markdown==3.7
+markdown-it-py==3.0.0
+MarkupSafe==3.0.2
+matplotlib==3.9.4
+mdurl==0.1.2
+ml-dtypes==0.4.1
+mpmath==1.3.0
+namex==0.0.8
+networkx==3.2.1
+numpy==2.0.2
+nvidia-cublas-cu12==12.4.5.8
+nvidia-cuda-cupti-cu12==12.4.127
+nvidia-cuda-nvrtc-cu12==12.4.127
+nvidia-cuda-runtime-cu12==12.4.127
+nvidia-cudnn-cu12==9.1.0.70
+nvidia-cufft-cu12==11.2.1.3
+nvidia-curand-cu12==10.3.5.147
+nvidia-cusolver-cu12==11.6.1.9
+nvidia-cusparse-cu12==12.3.1.170
+nvidia-cusparselt-cu12==0.6.2
+nvidia-nccl-cu12==2.21.5
+nvidia-nvjitlink-cu12==12.4.127
+nvidia-nvtx-cu12==12.4.127
+opt_einsum==3.4.0
+optree==0.14.1
+optuna==4.2.1
+optuna-integration==4.2.1
+packaging==24.2
+pandas==2.2.3
+pillow==11.1.0
+protobuf==5.29.3
+psutil==7.0.0
+Pygments==2.19.1
+pyparsing==3.2.1
+python-dateutil==2.9.0.post0
+pytz==2025.1
+PyYAML==6.0.2
+requests==2.32.3
+rich==13.9.4
+scikit-learn==1.6.1
+scipy==1.13.1
+seaborn==0.13.2
+Shimmy==2.0.0
+six==1.17.0
+SQLAlchemy==2.0.38
+stable_baselines3==2.5.0
+sympy==1.13.1
+tabulate==0.9.0
+tensorboard==2.18.0
+tensorboard-data-server==0.7.2
+tensorflow==2.18.0
+tensorflow-io-gcs-filesystem==0.37.1
+termcolor==2.5.0
+threadpoolctl==3.5.0
+torch==2.6.0
+tqdm==4.67.1
+triton==3.2.0
+typing_extensions==4.12.2
+tzdata==2025.1
+urllib3==2.3.0
+Werkzeug==3.1.3
+wrapt==1.17.2
+zipp==3.21.0