MidasTechnologiesInc/MidasEngine
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Trying to fix flake8

Browse Source
This commit is contained in:
klein panic
2024-12-13 02:43:47 -05:00
parent 4f1d2ab767
commit 03c46dcfb5
32 changed files with 2925 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
.flake8
View File

@@ -1,21 +1,19 @@
[flake8] [flake8]
max-line-length = 88 max-line-length = 88
exclude = # Directories to exclude from linting
venv, exclude =
.git, venv,
__pycache__, .git,
__pycache__
ignore = # Error codes to ignore
E203, # Whitespace before ':', compatible with Black formatting # E203, W503 are ignored for compatibility with Black formatting
W503, # Line break before binary operator, also compatible with Black formatting # E501 is ignored because max-line-length is set
E501, # Line length (use max-line-length instead) # F841 is useful for placeholders or debugging
E402, # Module level import not at top of file ignore =
F841, # Local variable assigned to but never used (useful for placeholders or debugging) E203,
W291, # Trailing whitespace W503,
W292, # No newline at end of file E501,
W293, # Blank line contains whitespace F841
E302, # Expected 2 blank lines, found 1 (can make code more compact if ignored)
E305, # Expected 2 blank lines after class or function definition
E741, # Ambiguous variable name (e.g., l, O)

18
.github/workflows/lint.yml vendored
View File

@@ -19,13 +19,25 @@ jobs:
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v2 uses: actions/setup-python@v2
with: with:
python-version: '3.x' python-version: '3.12' # Specify the exact Python version
- name: Cache pip dependencies
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies - name: Install dependencies
run: | run: |
python -m venv venv # Create a virtual environment in the project root python -m venv venv # Create a virtual environment in the project root
./venv/bin/pip install -r requirements.txt # Install dependencies from requirements.txt source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt # Install dependencies from requirements.txt
- name: Run Flake8 - name: Run Flake8
run: | run: |
./venv/bin/flake8 . --config=.flake8 # Run flake8 using the configuration in .flake8 source venv/bin/activate
flake8 . --config=.flake8 # Run flake8 using the configuration in .flake8

1
.gitignore vendored
View File

@@ -1 +1,2 @@
venv/ venv/
*.bak

1
README.md
View File

@@ -147,3 +147,4 @@ For the license file, please navigate to the docs/BusinessDocumentation/LICENSE
``` Author ``` Author
KleinPanic KleinPanic
``` ```

0
docs/ManPages/MidasV1Docs.md → docs/BusinessDocumentation/CodingPlans/MidasV1Docs.md
View File

0
docs/MidasTechNologiesLLCBylaws.md → docs/BusinessDocumentation/LegalDocs/MidasTechNologiesLLCBylaws.md
View File

3
docs/ManPages/README.md
View File

@@ -1,3 +0,0 @@
# ManPage Readme
- This directory will contain the actual man pages

233
docs/ManPages/midasv1.1 Normal file
View File

@@ -0,0 +1,233 @@
.TH MIDASV1 1 "December 2024" "MidasV1 Manual" "MidasV1 Manual"
.SH NAME
midasv1 \- A modular trading bot for Interactive Brokers
.SH SYNOPSIS
.B python
.RS
.B main.py
[\fB--no-checks\fP]
[\fB--skip-checks\fP]
[\fB--verbose\fP]
[\fB--version\fP]
.RE
.SH DESCRIPTION
MidasV1 is a sophisticated trading bot written in Python, designed to interact seamlessly with the Interactive Brokers (IB) Gateway or the IBJTS API and JTS. Leveraging a modular architecture, MidasV1 performs comprehensive system checks, retrieves and refines stock data, executes trading strategies based on real-time market analysis, and manages risks effectively.
.SH FEATURES
.IP \fBModular Architecture\fP
MidasV1 is divided into multiple modules, each responsible for distinct functionalities, ensuring a clear separation of concerns and streamlined operations.
.IP \fBComprehensive System Checks\fP
Before initiating trading operations, MidasV1 verifies the operating system, checks for necessary dependencies, assesses system resources, and ensures a secure connection with the IB Gateway.
.IP \fBReal-Time Market Data Retrieval\fP
Utilizes the IB API to scan and refine a list of stocks based on predefined criteria such as trading volume, net change, and volatility.
.IP \fBAutomated Trading Strategies\fP
Analyzes real-time market data to determine bullish or bearish trends and executes trades accordingly, incorporating risk management protocols.
.IP \fBRisk Management\fP
Implements multi-stage risk assessments to ensure trades are executed within acceptable risk parameters, including stop-loss mechanisms.
.SH WORKFLOW AND PROGRAM DESIGN
MidasV1 operates through a series of interconnected modules, each performing specific tasks:
.IP \fBModule 1: Initial Checks\fP
Includes:
.IP a. Operating System Check
Determines the operating system, supporting Linux by default. Unsupported OSes like Windows and MacOS trigger informative warnings and exit gracefully.
.IP b. Dependency Check
Reads required dependencies from `requirements.txt` located in the parent directory. Ensures all necessary Python packages are installed, informing the user of any missing dependencies with clear instructions.
.IP c. Connectivity Check
Verifies a secure connection with the IB Gateway using configuration parameters from `config/config.config`. Reports connectivity status with colored and formatted messages for enhanced readability.
.IP \fBModule 2: IBJTS List Petitioner\fP
Includes:
.IP a. Scanner
Retrieves a list of stocks that meet predefined criteria such as search volume, net change, and percent change.
.IP b. Refiner
Further refines the scanned stock list based on additional criteria like share price, availability of option contracts, and volatility index.
.IP \fBModule 3: Stock Information Retrieval\fP
Includes:
.IP a. Load
Loads the refined stock list from Module 2.
.IP b. Threaded Information Gathering & Choosing Strategy
Spawns individual threads to fetch real-time market data for each stock and determines the optimal trading strategy based on collected data.
.IP c. Strategy Implementation & Market Determination
Analyzes indicators like RSI, MACD, ADX, and EMA to classify market sentiment as bullish or bearish for each stock.
.IP \fBModule 4: Option Chain Trading & Risk Management\fP
Includes:
.IP a. Option Chain Data
Retrieves and analyzes option chain data for selected bullish and bearish stocks.
.IP b. Risk Management Stage 1
Assesses the acceptability of risk based on user account balance and predefined thresholds.
.IP c. Buying and Selling / Risk Management Stage 2
Executes trades, sets up stop-loss orders, and continuously monitors market data to implement selling strategies.
.IP \fBGeneral Additions\fP
- **Command-Line Flags:** Allows users to run the program with or without initial checks, enable verbose output, and display the program version.
- **Logging & Console Outputs:** Implements both file-based logging and colored console outputs for better traceability and user feedback.
- **Graceful Shutdowns:** Handles interrupt signals to ensure connections are closed properly.
- **Extensibility:** Designed to determine the number of threads based on system resources for optimal performance.
.SH FILE STRUCTURE
The project directory `MidasV1` is organized as follows:
.IP \fBproject_root/MidasV1/\fP
.RS
.IP \fBREADME.md\fP
Documentation and overview of the project.
.IP \fBrequirements.txt\fP
Lists all Python dependencies required by MidasV1.
.IP \fBconfig/config.config\fP
Configuration file containing all necessary parameters and thresholds.
.IP \fBmain.py\fP
The primary script that orchestrates the application's flow.
.IP \fBmodules/\fP
Contains all modular components of MidasV1.
.IP \fBmodules/initial_checks.py\fP
Performs system and environment checks.
.IP \fBmodules/stock_list_petitioner.py\fP
Retrieves and refines stock lists based on criteria.
.IP \fBtests/\fP
Contains test scripts for various modules.
.IP \fBtests/test_stock_retriever.py\fP
Tests stock retrieval and filtering.
.IP \fBtests/test_connection.py\fP
Tests connectivity with IB Gateway.
.IP \fBlogs/\fP
Stores detailed execution logs.
.IP \fBlogs/MidasV1.log\fP
.IP \fBdata/\fP
Stores JSON files with raw market data for each stock.
.RE
.SH INSTALLATION
.IP \fBPrerequisites\fP
.B Python 3.6+
.B Interactive Brokers (IB) Account
.IP \fBSteps\fP
.RS
.IP 1. Clone the Repository
.B
.RS
.nf
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
.RE
.IP 2. Download IB API
Download and install the latest version of the [Interactive Brokers API](https://www.interactivebrokers.com/en/index.php?f=5041).
.IP 3. Configure IB Gateway
Ensure that IB Gateway is running with API settings configured to allow connections from your machine.
.RE
.SH CONFIGURATION
All configurable parameters are stored in `config/config.config`. Below is an example configuration:
.RS
.IP \fB[Connectivity]\fP
.B host = 127.0.0.1
.B port = 4002
.B client_id = 0
.IP \fB[Logging]\fP
.B level = INFO
.IP \fB[Module2]\fP
.B default_search_volume = 1000000
.B default_net_change = 0.50
.B default_percent_change = 2.0
.B default_refinement_share_price = 15.0
.B default_volatility_threshold = 30.0
.B conditional_refinement_enabled = True
.B max_refined_list_size = 100
.RE
.SH USAGE
Run the application using the following command:
.RS
.IP \fBpython main.py\fP
[\fB--no-checks\fP]
[\fB--skip-checks\fP]
[\fB--verbose\fP]
[\fB--version\fP]
.RE
.IP \fBAvailable Flags\fP
.TP
\fB--no-checks\fP
Run the program without prompting for user confirmation after initial checks.
.TP
\fB--skip-checks\fP
Skip specific initial checks (primarily dependency checks).
.TP
\fB--verbose\fP
Enable verbose and colorful output to the console.
.TP
\fB--version\fP
Print the program version and exit.
.RE
.SH LOGGING
MidasV1 utilizes both file-based logging and colored console outputs to track its operations.
.IP \fBLog File\fP
.B logs/MidasV1.log
Stores detailed logs including debug information, errors, and informational messages.
.IP \fBConsole Outputs\fP
Color-coded messages enhance readability:
.IP \fBGreen\fP
Indicates successful operations or passing criteria.
.IP \fBRed\fP
Highlights errors or critical issues.
.IP \fBYellow\fP
Used for warnings and informational messages.
.IP \fBBlue\fP and \fBMagentafP
Used for decorative separators and headers.
.RE
.SH FUTURE ENHANCEMENTS
MidasV1 is designed with scalability in mind, allowing for future feature additions and optimizations:
.IP \fBOperating System Support\fP
Extend support to Windows, MacOS, BSD, illumos, etc., with specific handling mechanisms.
.IP \fBAdvanced Dependency Management\fP
Implement dynamic dependency resolution and version management.
.IP \fBEnhanced Strategy Module\fP
Develop more sophisticated trading strategies based on additional market indicators.
.IP \fBRisk Management Enhancements\fP
Implement multi-stage risk assessments and portfolio diversification strategies.
.IP \fBPerformance Optimization\fP
Utilize system resource checks to dynamically allocate threads for optimal performance.
.IP \fBUser Interface\fP
Develop a graphical user interface (GUI) for easier interaction and monitoring.
.IP \fBExtensive Testing\fP
Expand test coverage to include integration and stress tests, and implement CI pipelines.
.IP \fBDocumentation & Support\fP
Enhance documentation with tutorials and usage guides, and provide support mechanisms.
.RE
.SH DISCLAIMER
.B MidasV1
is proprietary software developed for private use. Unauthorized distribution, replication, or modification is strictly prohibited. The author assumes no responsibility for any misuse or damages resulting from the use of this software. Users are advised to thoroughly test the application in a controlled environment (e.g., paper trading) before deploying it in live trading scenarios.
.SH AUTHOR
.CC
kleinpainc (kleinpanic@gmail.com)
.RE
.SH COPYRIGHT
.CC
© 2024 kleinpanic. All rights reserved.
.RE

134
docs/README.md
View File

@@ -1,6 +1,6 @@
# Documentation Directory # Documentation Directory
Welcome to the **Documentation** directory for the project. This directory is divided into three primary sections: **Business Documentation**, **Policies and Standards**, and **Man Pages**. Each subdirectory has a unique purpose and serves to organize our project’s documentation comprehensively. Please follow the structure and guidelines below when adding new documents. Welcome to the **Documentation** directory for the **MidasV1** project. This directory is organized into three primary sections: **Business Documentation**, **Policies and Standards**, and **Man Pages**. Each subdirectory serves a unique purpose, ensuring comprehensive and organized documentation for our project. Please adhere to the structure and guidelines below when adding new documents.
--- ---
@@ -9,19 +9,44 @@ Welcome to the **Documentation** directory for the project. This directory is di
``` ```
docs/ docs/
│ │
├── business_docs/ ├── BusinessDocumentation/
│ ├── business_plan.md │ ├── BusinessPlans/
│ ├── lcc_documentation.md │ │ ├── AUP.md
│ └── legal_license.md │ │ ├── ConfidentialityAgreementForThirdPartyContractorsAndVendors.md
│ │ ├── ContributorAgreement.md
│ │ ├── CyberSecurityAgreement.md
│ │ ├── DataRetentionPolicy.md
│ │ ├── DataUseAdPrivacyPolicy.md
│ │ ├── DisasterRecoveryAndBusinessContinuityPlan.md
│ │ ├── EmployeeConsultantOnboardingAgreement.md
│ │ ├── EmployeeHandbook.md
│ │ ├── EULA.md
│ │ ├── ExecutiveSummary.md
│ │ ├── IntellectualPropertyAssignmentAgreement.md
│ │ ├── Non-CompeteAndNon-SolicitationAgreement.md
│ │ ├── oil_oracle.md
│ │ ├── OperatingAgreement.md
│ │ ├── PartnershipAgreement.md
│ │ ├── SoftwareDevelopmentAndLicensingAgreements.md
│ │ └── TradeSecretPolicy.md
│ ├── CodingPlans/
│ │ └── MidasV1Docs.md
│ ├── LegalDocs/
│ │ └── MidasTechNologiesLLCBylaws.md
│ └── LICENSE
│ │
├── policies_standards/ ├── ManPages/
│ ├── coding_standards.md │ └── midasv1.1
│ ├── code_review_policy.md
│ ├── documentation_standards.md
│ └── file_path_standards.md
│ │
└── man_pages/ ├── PoliciesAndStandards/
└── overall_code_documentation.md │ ├── CodingStandards.md
│ ├── CommunicationStandards.md
│ ├── DocumentationStandards.md
│ ├── FilePathStandards.md
│ ├── GitAndGithubStandards.md
│ └── README.md
│
└── README.md
``` ```
--- ---
@@ -30,26 +55,52 @@ docs/
### 1. Business Documentation ### 1. Business Documentation
- **Purpose**: Contains all relevant business-related documentation, legal filings, and operational plans. This section helps ensure that we keep a clear record of our company’s structural and strategic information. - **Purpose**: Contains all business-related documentation, legal filings, and operational plans. This section ensures a clear record of our company’s structural and strategic information.
- **Contents**: - **Contents**:
- `business_plan.md`: Outlines our company’s mission, vision, target market, competitive analysis, and growth strategies. - `BusinessPlans/`:
- `lcc_documentation.md`: Contains LLC formation details, ownership structure, and any legal requirements. - **AUP.md**: Acceptable Use Policy outlining permitted and prohibited activities.
- `legal_license.md`: Provides a summary of licenses, terms, and legal disclaimers for our software products. - **ConfidentialityAgreementForThirdPartyContractorsAndVendors.md**: Agreements ensuring confidentiality with third-party contractors and vendors.
- **ContributorAgreement.md**: Terms and conditions for contributors to the project.
- **CyberSecurityAgreement.md**: Policies related to cybersecurity measures and protocols.
- **DataRetentionPolicy.md**: Guidelines on data retention and disposal.
- **DataUseAdPrivacyPolicy.md**: Policies governing data usage and privacy.
- **DisasterRecoveryAndBusinessContinuityPlan.md**: Plans for disaster recovery and maintaining business operations during disruptions.
- **EmployeeConsultantOnboardingAgreement.md**: Agreements for onboarding employees and consultants.
- **EmployeeHandbook.md**: Comprehensive guide for employees covering company policies, procedures, and culture.
- **EULA.md**: End-User License Agreement detailing usage terms for software products.
- **ExecutiveSummary.md**: High-level overview of the company's mission, vision, and strategic goals.
- **IntellectualPropertyAssignmentAgreement.md**: Agreements ensuring assignment of intellectual property rights.
- **Non-CompeteAndNon-SolicitationAgreement.md**: Agreements preventing competition and solicitation of clients or employees.
- **oil_oracle.md**: Specific documentation related to the Oil Oracle project/component.
- **OperatingAgreement.md**: Operational guidelines for the company’s management.
- **PartnershipAgreement.md**: Agreements detailing the terms of partnerships.
- **SoftwareDevelopmentAndLicensingAgreements.md**: Agreements related to software development and licensing.
- **TradeSecretPolicy.md**: Policies protecting trade secrets and sensitive information.
- `CodingPlans/`:
- **MidasV1Docs.md**: Documentation related to the coding plans and strategies for MidasV1.
- `LegalDocs/`:
- **MidasTechNologiesLLCBylaws.md**: Bylaws governing the operations of Midas Technologies LLC.
- `LICENSE`: Licensing information for the project.
### 2. Policies and Standards ### 2. Policies and Standards
- **Purpose**: This section includes all coding, collaboration, and workflow guidelines to maintain consistency and quality across the project. It also contains standard operating procedures (SOPs) for code quality and collaboration, fostering a productive and unified development environment. - **Purpose**: Includes all coding, collaboration, and workflow guidelines to maintain consistency and quality across the project. It also contains standard operating procedures (SOPs) for code quality and collaboration, fostering a productive and unified development environment.
- **Contents**: - **Contents**:
- `coding_standards.md`: Outlines coding conventions and best practices for this project. - `CodingStandards.md`: Outlines coding conventions and best practices for this project.
- `code_review_policy.md`: Details our policies for reviewing code and handling pull requests. - `CommunicationStandards.md`: Defines standards for team communication and collaboration.
- `documentation_standards.md`: Specifies standards for documenting code and creating README files. - `DocumentationStandards.md`: Specifies standards for documenting code and creating README files.
- `file_path_standards.md`: Establishes naming conventions and organization standards for files and directories. - `FilePathStandards.md`: Establishes naming conventions and organization standards for files and directories.
- `GitAndGithubStandards.md`: Details best practices for using Git and GitHub, including branching strategies and commit messages.
- `README.md`: Overview and guidelines specific to the Policies and Standards section.
### 3. Man Pages ### 3. Man Pages
- **Purpose**: Acts as an accessible, high-level guide to overall code structure, intended to assist both new and existing team members in understanding the project’s architecture. This section contains **man pages** for broader code documentation, while specific functionality and module-level details can be found in `src/` README files. - **Purpose**: Provides accessible, high-level guides and detailed documentation for the codebase. This section contains **man pages** for broader code documentation, assisting both new and existing team members in understanding the project’s architecture and functionalities.
- **Contents**: - **Contents**:
- `overall_code_documentation.md`: Offers a high-level overview of the codebase, including descriptions of major components, workflows, and integrations within the project. - `midasv1.1`: The manual page for MidasV1, detailing its usage, options, and functionalities. This man page can be accessed using the `man` command.
--- ---
@@ -57,13 +108,48 @@ docs/
1. **Check the Directory**: 1. **Check the Directory**:
- Ensure you are adding files to the correct subdirectory according to the document’s purpose. - Ensure you are adding files to the correct subdirectory according to the document’s purpose.
- **Business Documentation**: Add business-related and legal documents.
- **Policies and Standards**: Add coding, communication, and workflow policies.
- **Man Pages**: Add manual pages and advanced code documentation.
2. **Naming Conventions**: 2. **Naming Conventions**:
- Use clear, concise, and descriptive file names for new documents. - Use clear, concise, and descriptive file names for new documents.
- File names should be lowercase and use underscores instead of spaces (e.g., `new_policy_doc.md`). - File names should be lowercase with underscores instead of spaces (e.g., `new_policy_doc.md`).
3. **Update Section Summaries**: 3. **Update Section Summaries**:
- When adding new files, briefly update this `README.md` to reflect any changes within each section. - When adding new files, briefly update this `README.md` to reflect any changes within each section.
- Ensure consistency in documentation style and formatting across all files.
4. **Maintain Consistency**:
- Follow existing structures and templates when creating new documentation.
- Adhere to the standards outlined in the `PoliciesAndStandards` section to ensure uniformity.
5. **Review Before Adding**:
- Before adding a new document, verify that it does not duplicate existing content.
- Ensure that all new documentation is reviewed for accuracy and completeness.
--- ---
Each contributor should refer to this directory and structure before adding or modifying documentation to maintain clarity and organization throughout the project. For specific questions on policies or code documentation, refer to the relevant files in **Policies and Standards** or **Man Pages**. Each contributor should refer to this directory and structure before adding or modifying documentation to maintain clarity and organization throughout the project. For specific questions on policies or code documentation, refer to the relevant files in **Policies and Standards** or **Man Pages**.
---
## Additional Documentation
Beyond the primary sections, the `docs/` directory may include additional documentation as needed. Ensure that any new documentation aligns with the existing structure and adheres to the guidelines outlined above.
---
**Note**: The **Man Pages** section is dedicated to actual manpages and advanced code documentation. Ensure that manpages are formatted correctly using the appropriate tools (e.g., `groff`) to maintain readability and functionality when accessed via the `man` command.
---
## Contact
For any questions or further assistance regarding the documentation structure or adding new documents, please reach out to the project maintainer.
---
## License
Please refer to the `LICENSE` file within the `BusinessDocumentation` directory for licensing information pertaining to the MidasV1 project.

0
src/Data-Collection/WebScraper/data/HistoricalData.json → src/Data-Collection/HistoricalData.json
View File

2
src/Data-Collection/WebScraper/data/preprocessed_oil_news.json
View File

@@ -4000,4 +4000,4 @@
] ]
] ]
} }
] ]

478
src/MidasV1/README.md Normal file
View File

@@ -0,0 +1,478 @@
# MidasV1
![MidasV1 Logo](https://via.placeholder.com/150) <!-- Replace with actual logo if available -->
## Table of Contents
- [Overview](#overview)
- [General Technical Overview](#general-technical-overview)
- [Workflow & Program Design](#workflow--program-design)
- [Module 1: Initial Checks](#module-1-initial-checks)
- [a. Operating System Check](#a-operating-system-check)
- [b. Dependency Check](#b-dependency-check)
- [c. Connectivity Check](#c-connectivity-check)
- [Module 2: IBJTS List Petitioner](#module-2-ibjts-list-petitioner)
- [a. Scanner](#a-scanner)
- [b. Refiner](#b-refiner)
- [Module 3: Stock Information Retrieval](#module-3-stock-information-retrieval)
- [a. Load](#a-load)
- [b. Threaded Information Gathering & Choosing Strategy](#b-threaded-information-gathering--choosing-strategy)
- [c. Strategy Implementation & Market Determination](#c-strategy-implementation--market-determination)
- [Module 4: Option Chain Trading & Risk Management](#module-4-option-chain-trading--risk-management)
- [a. Option Chain Data](#a-option-chain-data)
- [b. Risk Management Stage 1](#b-risk-management-stage-1)
- [c. Buying and Selling / Risk Management Stage 2](#c-buying-and-selling--risk-management-stage-2)
- [General Additions](#general-additions)
- [File Structure](#file-structure)
- [Installation](#installation)
- [Configuration](#configuration)
- [Usage](#usage)
- [Logging](#logging)
- [Future Enhancements](#future-enhancements)
- [Disclaimer](#disclaimer)
---
## Overview
**MidasV1** is a sophisticated trading bot developed in Python, designed to interact seamlessly with the Interactive Brokers (IB) Gateway or the IBJTS API and JTS. Leveraging a modular architecture, MidasV1 performs comprehensive system checks, retrieves and refines stock data, executes trading strategies based on real-time market analysis, and manages risks effectively.
This README provides an in-depth overview of MidasV1's architecture, functionalities, and setup instructions.
---
## General Technical Overview
- **Programming Language:** Python
- **Dependencies:**
- `ibapi`: Interactive Brokers API for Python
- `psutil`: For system resource monitoring
- **Requirements:**
- **IBJTS API & JTS:** Ensure that the IBJTS API and JTS are running.
- **IB Gateway:** Alternatively, IB Gateway can be used for connectivity.
- **Architecture:** Highly modular, facilitating scalability and maintainability.
---
## Workflow & Program Design
MidasV1 is structured into multiple modules, each responsible for distinct functionalities, ensuring a clear separation of concerns and streamlined operations.
### Module 1: Initial Checks
#### a. Operating System Check
- **Purpose:** Determine the operating system of the device running MidasV1.
- **Supported OS:** Linux
- **Unsupported OS:** Windows, MacOS, BSD, illumos, etc.
- **Behavior:**
- **Linux:** Continues with execution, providing a green success message.
- **Unsupported OSes:** Displays red error messages and yellow warnings indicating future support considerations, then gracefully exits.
#### b. Dependency Check
- **Purpose:** Ensure all necessary Python packages are installed.
- **Mechanism:**
- **Source:** Reads dependencies from `requirements.txt` located in the parent directory of the `modules` folder.
- **Process:**
- Parses `requirements.txt` to extract package names.
- Checks if each package is installed using `pkg_resources`.
- **Missing Dependencies:** Informs the user with red text and provides the exact `pip` command to install them.
- **All Dependencies Present:** Confirms with a green success message.
#### c. Connectivity Check
- **Purpose:** Verify a secure and functional connection with the IB Gateway.
- **Configuration:**
- **Source:** Retrieves `host`, `port`, and `client_id` from `config/config.config` with default fallbacks (`127.0.0.1`, `4002`, `0` respectively).
- **Behavior:**
- Attempts to establish a connection to the IB Gateway.
- **Success:** Displays green messages confirming connection and successful retrieval of account summaries.
- **Failure:** Shows red error messages and yellow warnings, then exits gracefully.
- **Enhancements:**
- Utilizes colored console outputs for clear and human-readable reporting.
- Addresses the unused `advancedOrderRejectJson` parameter to eliminate warnings.
### Module 2: IBJTS List Petitioner
#### a. Scanner
- **Purpose:** Retrieve a list of stocks that meet predefined criteria.
- **Configuration:** Loads criteria such as search volume, net change, and percent change from `config/config.config`.
- **Process:**
- Initiates a scanner subscription via the IB API using the loaded criteria.
- Requests a list of stocks that satisfy the specified metrics.
- Caches the retrieved list temporarily for further processing.
#### b. Refiner
- **Purpose:** Further refine the scanned stock list based on additional criteria.
- **Criteria:**
1. **Share Price:** Exclude stocks with share prices exceeding a threshold defined in the config.
2. **Option Contracts:** Remove stocks without available option contracts.
3. **Volatility Index:** Exclude stocks with a volatility index above the configured threshold.
4. **Conditional Truncation:** If enabled, truncate the list to a maximum size specified in the config.
- **Behavior:**
- Applies each refinement step sequentially.
- Provides colored console outputs indicating inclusion or exclusion of stocks.
- Caches the refined list for transfer to subsequent modules.
### Module 3: Stock Information Retrieval
#### a. Load
- **Purpose:** Load the refined stock list from Module 2.
- **Behavior:** Ensures the availability of the refined list and prepares it for data retrieval.
#### b. Threaded Information Gathering & Choosing Strategy
- **Purpose:** Gather real-time market data for each stock and determine the optimal trading strategy.
- **Process:**
- **Threading:** Spawns individual threads for each stock to fetch data asynchronously.
- **Data Retrieved:** Datetime, high, low, close, and volume at specified trading intervals.
- **Data Storage:** Saves data in JSON files named `{stock_name}.{current_date}.json` within the `data/` directory.
- **Strategy Counter:** Maintains a counter based on incoming data to determine trend indicators.
#### c. Strategy Implementation & Market Determination
- **Purpose:** Analyze collected data to identify bullish or bearish trends.
- **Indicators:**
- **RSI (Relative Strength Index)**
- **MACD (Moving Average Convergence Divergence)**
- **ADX (Average Directional Index)**
- **EMA (Exponential Moving Average)**
- **Behavior:**
- Calculates or retrieves indicator values.
- Assigns weights to each indicator based on predefined thresholds.
- Determines overall market sentiment (bullish/bearish) for each stock.
- Based on internal boolean flags, decides whether to process the entire list or isolate the most bullish and bearish stocks for further actions.
### Module 4: Option Chain Trading & Risk Management
#### a. Option Chain Data
- **Purpose:** Retrieve and analyze option chain data for selected stocks.
- **Process:**
- **Data Retrieval:** Fetches option contracts closest to the current share price.
- **Filtering:**
- **Bearish Stocks:** Isolates contracts with strike prices slightly above the share price.
- **Bullish Stocks:** Isolates contracts with strike prices slightly below the share price.
- **Behavior:** Ensures that contracts are selected based on proximity to the current market price and other configurable parameters.
#### b. Risk Management Stage 1
- **Purpose:** Assess the acceptability of risk before executing trades.
- **Process:**
- Retrieves user account balance information.
- Determines if the cost of option contracts is within the acceptable risk percentage defined in the config.
- **Outcome:** Only proceeds with contracts that meet the risk criteria.
#### c. Buying and Selling / Risk Management Stage 2
- **Purpose:** Execute trades and manage ongoing risk.
- **Process:**
- **Trade Execution:** Buys option contracts that passed risk assessments.
- **Stop-Loss Orders:** Sets up stop-loss contracts based on configurable loss thresholds.
- **Continuous Monitoring:** Gathers real-time data to implement selling strategies, ensuring optimal trade exits.
### General Additions
- **Command-Line Flags:**
- `--no-checks`: Runs the program without prompting for user confirmation after initial checks.
- `--skip-checks`: Skips specific initial checks (primarily dependency checks).
- `--verbose`: Enables verbose and colorful output to the console.
- `--version`: Prints the program version and exits.
- **Logging & Console Outputs:**
- Implements both logging to files and colored console outputs.
- Controlled via the `--verbose` flag to manage verbosity levels.
- **Graceful Shutdowns:**
- Handles interrupt signals (e.g., Ctrl+C) to ensure connections are closed properly.
- **Extensibility:**
- Designed to determine the number of threads based on system resources for optimal performance.
---
## File Structure
```
MidasV1/
├── README.md
├── requirements.txt
├── config/
│ └── config.config
├── main.py
├── modules/
│ ├── initial_checks.py
│ └── stock_list_petitioner.py
├── tests/
│ ├── test_data_retriever.py
│ ├── test_stock_retriever.py
│ └── test_connection.py
├── logs/
│ └── MidasV1.log
└── data/
└── {stock_name}.{current_date}.json
```
- **README.md:** This documentation file.
- **requirements.txt:** Lists all Python dependencies required by MidasV1.
- **config/config.config:** Configuration file containing all necessary parameters and thresholds.
- **main.py:** The primary script that orchestrates the application's flow.
- **modules/:** Contains all modular components of MidasV1.
- **initial_checks.py:** Performs system and environment checks.
- **stock_list_petitioner.py:** Retrieves and refines stock lists based on criteria.
- **tests/:** Contains test scripts for various modules.
- **test_data_retriever.py:** Tests data retrieval functionalities.
- **test_stock_retriever.py:** Tests stock retrieval and filtering.
- **test_connection.py:** Tests connectivity with IB Gateway.
- **logs/MidasV1.log:** Logs detailed execution information.
- **data/:** Stores JSON files with raw market data for each stock.
---
## Installation
### Prerequisites
- **Python 3.6 or higher:** Ensure Python is installed on your system. You can verify the installation by running:
```bash
python --version
```
- **Interactive Brokers (IB) Account:** Required to access the IB Gateway or IBJTS API.
### Steps
1. **Clone the Repository:**
```bash
git clone https://github.com/yourusername/MidasV1.git
cd MidasV1
```
2. **Set Up Virtual Environment (Recommended):**
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
```
3. **Install Dependencies:**
```bash
pip install -r requirements.txt
```
4. **Download IB API:**
- **From Interactive Brokers:**
- Visit the [Interactive Brokers API](https://www.interactivebrokers.com/en/index.php?f=5041) download page.
- Download and install the latest version of the IB API.
- **IB Gateway:**
- Alternatively, install the [IB Gateway](https://www.interactivebrokers.com/en/index.php?f=16457) for a lightweight connection.
5. **Configure IB Gateway:**
- Launch IB Gateway and log in using your IB credentials.
- Ensure that the API settings allow connections from your machine:
- **Enable API:** Navigate to `Configure` > `Settings` > `API` > `Settings`.
- **Trusted IPs:** Add `127.0.0.1` or your specific IP address.
- **Port:** Ensure it matches the `port` specified in `config/config.config` (default is `4002`).
6. **Verify Configuration:**
- Ensure `config/config.config` is properly set with your desired parameters.
- Example configuration is provided below.
---
## Configuration
All configurable parameters are stored in `config/config.config`. Below is an example of the configuration file:
```ini
[General]
version = 1.0.0
# Future general configurations can be added here
[Connectivity]
host = 127.0.0.1
port = 4002
client_id = 0
# Add more connectivity parameters as needed
[SystemResources]
# Placeholder for system resource related configurations
# Example:
# max_cpu_threads = 8
# min_available_ram_gb = 4
[Logging]
level = INFO
# Available levels: DEBUG, INFO, WARNING, ERROR, CRITICAL
# Set to DEBUG for verbose logging, INFO for standard logging, etc.
[Module2]
default_search_volume = 1000000
default_net_change = 0.50
default_percent_change = 2.0
default_refinement_share_price = 15.0
default_volatility_threshold = 30.0
conditional_refinement_enabled = True
max_refined_list_size = 100
```
### Configuration Sections
- **[General]:** General settings, including the version of MidasV1.
- **[Connectivity]:**
- **host:** IP address of the IB Gateway (default `127.0.0.1`).
- **port:** Port number for the API connection (default `4002` for IB Gateway Simulated Trading).
- **client_id:** Unique client ID for the API connection.
- **[SystemResources]:**
- Placeholder for future configurations related to system resources.
- Example parameters for thread management can be added here.
- **[Logging]:**
- **level:** Logging verbosity (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`).
- **[Module2]:**
- **default_search_volume:** Minimum trading volume for stock selection.
- **default_net_change:** Minimum net change in stock price.
- **default_percent_change:** Minimum percentage change in stock price.
- **default_refinement_share_price:** Maximum share price threshold.
- **default_volatility_threshold:** Maximum acceptable volatility index.
- **conditional_refinement_enabled:** Boolean to enable or disable list truncation.
- **max_refined_list_size:** Maximum number of stocks in the refined list.
---
## Usage
### Running the Application
Navigate to the project root directory and execute `main.py` with desired flags:
```bash
python main.py [--no-checks] [--skip-checks] [--verbose] [--version]
```
### Available Flags
- `--no-checks`: Run the program without prompting for user confirmation after initial checks.
- `--skip-checks`: Skip specific initial checks (primarily dependency checks).
- `--verbose`: Enable verbose and colorful output to the console.
- `--version`: Print the program version and exit.
### Example Commands
1. **Standard Execution:**
```bash
python main.py
```
2. **Verbose Mode:**
```bash
python main.py --verbose
```
3. **Skip Dependency Checks:**
```bash
python main.py --skip-checks
```
4. **Run Without User Confirmation:**
```bash
python main.py --no-checks
```
5. **Display Version:**
```bash
python main.py --version
```
### Testing Modules
MidasV1 includes several test scripts located in the `tests/` directory to verify the functionality of individual modules.
1. **Test Stock Retriever:**
```bash
python tests/test_stock_retriever.py
```
2. **Test Connection:**
```bash
python tests/test_connection.py
```
3. **Test Data Retrieval:**
```bash
python tests/test_data_retriever.py
```
*Note: Ensure that the IB Gateway or IBJTS API is running before executing test scripts.*
---
## Logging
MidasV1 utilizes both file-based logging and console outputs to track its operations.
- **Log File:** `logs/MidasV1.log`
- **Location:** Stored in the `logs/` directory within the project root.
- **Content:** Detailed logs including debug information, errors, and informational messages.
- **Configuration:** Controlled via the `[Logging]` section in `config/config.config`.
- **Console Outputs:**
- **Color-Coded Messages:** Enhances readability with green for successes, red for errors, yellow for warnings, and blue/magenta for informational and decorative messages.
- **Verbosity:** Managed via the `--verbose` flag and the logging level set in the configuration file.
*Ensure that the `logs/` directory exists or is created before running the application to prevent logging errors.*
---
## Future Enhancements
MidasV1 is designed with scalability in mind, allowing for future feature additions and optimizations.
1. **Operating System Support:**
- Extend support to Windows, MacOS, BSD, illumos, etc., with specific handling mechanisms.
2. **Advanced Dependency Management:**
- Implement dynamic dependency resolution and version management.
3. **Enhanced Strategy Module:**
- Develop more sophisticated trading strategies based on additional market indicators.
- Incorporate machine learning algorithms for predictive analysis.
4. **Risk Management Enhancements:**
- Implement multi-stage risk assessments.
- Integrate portfolio diversification strategies.
5. **Performance Optimization:**
- Utilize system resource checks to dynamically allocate threads for optimal performance.
- Implement rate limiting and efficient data handling mechanisms.
6. **User Interface:**
- Develop a graphical user interface (GUI) for easier interaction and monitoring.
- Provide real-time dashboards for tracking trades and system status.
7. **Extensive Testing:**
- Expand test coverage to include integration and stress tests.
- Implement continuous integration (CI) pipelines for automated testing.
8. **Documentation & Support:**
- Enhance documentation with tutorials and usage guides.
- Provide support mechanisms for troubleshooting and user assistance.
---
## Disclaimer
**MidasV1** is proprietary software developed for private use. Unauthorized distribution, replication, or modification is strictly prohibited. The author assumes no responsibility for any misuse or damages resulting from the use of this software. Users are advised to thoroughly test the application in a controlled environment (e.g., paper trading) before deploying it in live trading scenarios.
---
## Additional Information
- **Contact:** For inquiries or support, please contact [kleinpanic@gmail.com](mailto:kleinpanic@gmail.com).
- **License:** All rights reserved. No part of this software may be reproduced or transmitted in any form or by any means without the prior written permission of the author.
---

32
src/MidasV1/config/config.config Normal file
View File

@@ -0,0 +1,32 @@
# config/config.config
[General]
version = 1.0.0
# Future general configurations can be added here
[Connectivity]
host = 127.0.0.1
port = 4002
client_id = 0
# Add more connectivity parameters as needed
[SystemResources]
# Placeholder for system resource related configurations
# Example:
# max_cpu_threads = 8
# min_available_ram_gb = 4
[Logging]
level = INFO
# Available levels: DEBUG, INFO, WARNING, ERROR, CRITICAL
# Set to DEBUG for verbose logging, INFO for standard logging, etc.
[Module2]
default_search_volume = 1000000
default_net_change = 0.50
default_percent_change = 2.0
default_refinement_share_price = 15.0
default_volatility_threshold = 30.0
conditional_refinement_enabled = True
max_refined_list_size = 100

164
src/MidasV1/logs/MidasV1.log Normal file
View File

@@ -0,0 +1,164 @@
2024-12-13 01:25:45,907 - MidasV1.Main - INFO - Starting MidasV1 Trading Bot...
2024-12-13 01:25:45,907 - MidasV1.InitialChecks - INFO - Checking Operating System...
2024-12-13 01:25:45,907 - MidasV1.InitialChecks - INFO - Operating System Linux is supported.
2024-12-13 01:25:45,907 - MidasV1.InitialChecks - INFO - Checking Dependencies...
2024-12-13 01:25:45,907 - MidasV1.InitialChecks - INFO - All dependencies are satisfied.
2024-12-13 01:25:45,907 - MidasV1.InitialChecks - INFO - Checking System Resources...
2024-12-13 01:25:45,908 - MidasV1.InitialChecks - INFO -
CPU Cores: 12
CPU Clock Speed: 891.19 MHz
CPU Load Average (1m, 5m, 15m): (0.333984375, 0.28125, 0.21630859375)
CPU Threads: 12
Total RAM: 15.31 GB
Used RAM: 3.15 GB (27.7%)
Available RAM: 11.08 GB
Total Swap: 0.95 GB
Used Swap: 0.00 GB (0.0%)
2024-12-13 01:25:45,908 - MidasV1.InitialChecks - INFO - Checking Connectivity with IB Gateway...
2024-12-13 01:25:45,908 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 01:25:45,908 - MidasV1.InitialChecks - INFO - IB Gateway Connection Test
2024-12-13 01:25:45,908 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 01:25:45,908 - MidasV1.InitialChecks - INFO - Attempting to connect to IB Gateway at 127.0.0.1:4002...
2024-12-13 01:25:45,912 - MidasV1.InitialChecks - INFO - [INFO] Managed accounts: DUE064818
2024-12-13 01:25:45,912 - MidasV1.InitialChecks - INFO - [INFO] Next valid order ID: 1
2024-12-13 01:25:45,912 - MidasV1.InitialChecks - INFO - [INFO/STATUS] id=-1, code=2104, msg=Market data farm connection is OK:usfarm
2024-12-13 01:25:45,955 - MidasV1.InitialChecks - INFO - [INFO/STATUS] id=-1, code=2107, msg=HMDS data farm connection is inactive but should be available upon demand.ushmds
2024-12-13 01:25:45,955 - MidasV1.InitialChecks - INFO - [INFO/STATUS] id=-1, code=2158, msg=Sec-def data farm connection is OK:secdefnj
2024-12-13 01:25:46,011 - MidasV1.InitialChecks - INFO - [INFO] Connected successfully!
2024-12-13 01:25:46,011 - MidasV1.InitialChecks - INFO - Requesting account summary...
2024-12-13 01:25:46,041 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, BuyingPower = 4014612.28 USD
2024-12-13 01:25:46,042 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, EquityWithLoanValue = 1003653.07 USD
2024-12-13 01:25:46,042 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, NetLiquidation = 1004890.64 USD
2024-12-13 01:25:46,043 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, TotalCashValue = 1003653.07 USD
2024-12-13 01:25:46,043 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY END] ReqId: 1
2024-12-13 01:25:51,012 - MidasV1.InitialChecks - INFO - [INFO] Successfully retrieved account summary data.
2024-12-13 01:25:51,012 - MidasV1.InitialChecks - INFO - IB Gateway is connected and ready for upcoming modules.
2024-12-13 01:25:51,013 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 01:25:51,013 - MidasV1.InitialChecks - INFO - Test Complete
2024-12-13 01:25:51,013 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 01:25:51,013 - MidasV1.InitialChecks - INFO - All initial checks passed successfully.
2024-12-13 01:25:51,013 - MidasV1.StockListPetitioner - INFO - Starting Module 2: IBJTS List Petitioner...
2024-12-13 01:25:51,013 - MidasV1.StockListPetitioner - INFO - Loaded Scanner Criteria:
- Search Volume: 1000000
- Net Change: 0.5
- Percent Change: 2.0
2024-12-13 01:25:51,013 - MidasV1.StockListPetitioner - INFO - Initiating scanner subscription with the above criteria...
2024-12-13 01:25:51,014 - MidasV1.StockListPetitioner - INFO - Scanner subscription requested successfully on attempt 1.
2024-12-13 01:25:51,014 - MidasV1.StockListPetitioner - INFO - Waiting for scanner data (timeout in 15 seconds)...
2024-12-13 01:25:51,115 - MidasV1.InitialChecks - ERROR - [ERROR] id=1001, code=162, msg=Historical Market Data Service error message:Scanner type with code ALL is disabled
2024-12-13 01:26:06,014 - MidasV1.StockListPetitioner - ERROR - Scanner subscription timed out.
2024-12-13 01:26:06,015 - MidasV1.StockListPetitioner - ERROR - Scanner subscription canceled due to timeout.
2024-12-13 01:26:06,015 - MidasV1.Main - INFO - Refined Stock List: []
2024-12-13 01:26:06,015 - MidasV1.Main - INFO - Initial checks and Module 2 completed. Please review the logs and ensure everything is correct.
2024-12-13 01:26:06,017 - MidasV1.InitialChecks - ERROR - [ERROR] id=1001, code=365, msg=No scanner subscription found for ticker id:1001
2024-12-13 01:26:08,651 - MidasV1.Main - ERROR - Interrupt received. Shutting down gracefully...
2024-12-13 01:26:08,651 - MidasV1.InitialChecks - ERROR - [ERROR] Connection to IB Gateway was closed unexpectedly!
2024-12-13 01:26:08,651 - MidasV1.Main - INFO - Disconnected from IB Gateway.
2024-12-13 01:40:53,290 - MidasV1.Main - INFO - Starting MidasV1 Trading Bot...
2024-12-13 01:40:53,290 - MidasV1.InitialChecks - INFO - Checking Operating System...
2024-12-13 01:40:53,290 - MidasV1.InitialChecks - INFO - Operating System Linux is supported.
2024-12-13 01:40:53,290 - MidasV1.InitialChecks - INFO - Checking Dependencies...
2024-12-13 01:40:53,290 - MidasV1.InitialChecks - INFO - All dependencies are satisfied.
2024-12-13 01:40:53,290 - MidasV1.InitialChecks - INFO - Checking System Resources...
2024-12-13 01:40:53,291 - MidasV1.InitialChecks - INFO -
CPU Cores: 12
CPU Clock Speed: 1745.26 MHz
CPU Load Average (1m, 5m, 15m): (0.775390625, 0.732421875, 0.5390625)
CPU Threads: 12
Total RAM: 15.31 GB
Used RAM: 2.93 GB (26.5%)
Available RAM: 11.26 GB
Total Swap: 0.95 GB
Used Swap: 0.00 GB (0.0%)
2024-12-13 01:40:53,291 - MidasV1.InitialChecks - INFO - Checking Connectivity with IB Gateway...
2024-12-13 01:40:53,291 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 01:40:53,291 - MidasV1.InitialChecks - INFO - IB Gateway Connection Test
2024-12-13 01:40:53,291 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 01:40:53,291 - MidasV1.InitialChecks - INFO - Attempting to connect to IB Gateway at 127.0.0.1:4002...
2024-12-13 01:40:53,335 - MidasV1.InitialChecks - INFO - [INFO] Managed accounts: DUE064818
2024-12-13 01:40:53,379 - MidasV1.InitialChecks - INFO - [INFO] Next valid order ID: 1
2024-12-13 01:40:53,379 - MidasV1.InitialChecks - INFO - [INFO/STATUS] id=-1, code=2104, msg=Market data farm connection is OK:usfarm
2024-12-13 01:40:53,379 - MidasV1.InitialChecks - INFO - [INFO/STATUS] id=-1, code=2106, msg=HMDS data farm connection is OK:ushmds
2024-12-13 01:40:53,379 - MidasV1.InitialChecks - INFO - [INFO/STATUS] id=-1, code=2158, msg=Sec-def data farm connection is OK:secdefnj
2024-12-13 01:40:53,429 - MidasV1.InitialChecks - INFO - [INFO] Connected successfully!
2024-12-13 01:40:53,430 - MidasV1.InitialChecks - INFO - Requesting account summary...
2024-12-13 01:40:53,591 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, BuyingPower = 4014612.28 USD
2024-12-13 01:40:53,592 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, EquityWithLoanValue = 1003653.07 USD
2024-12-13 01:40:53,592 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, NetLiquidation = 1004890.64 USD
2024-12-13 01:40:53,593 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, TotalCashValue = 1003653.07 USD
2024-12-13 01:40:53,594 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY END] ReqId: 1
2024-12-13 01:40:58,431 - MidasV1.InitialChecks - INFO - [INFO] Successfully retrieved account summary data.
2024-12-13 01:40:58,432 - MidasV1.InitialChecks - INFO - IB Gateway is connected and ready for upcoming modules.
2024-12-13 01:40:58,432 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 01:40:58,432 - MidasV1.InitialChecks - INFO - Test Complete
2024-12-13 01:40:58,433 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 01:40:58,433 - MidasV1.InitialChecks - INFO - All initial checks passed successfully.
2024-12-13 01:40:58,433 - MidasV1.StockListPetitioner - INFO - Starting Module 2: IBJTS List Petitioner...
2024-12-13 01:40:58,433 - MidasV1.StockListPetitioner - INFO - Loaded Scanner Criteria:
- Search Volume: 1000000
- Net Change: 0.5
- Percent Change: 2.0
2024-12-13 01:40:58,434 - MidasV1.StockListPetitioner - INFO - Initiating scanner subscription with the above criteria...
2024-12-13 01:40:58,434 - MidasV1.StockListPetitioner - INFO - Scanner subscription requested successfully on attempt 1.
2024-12-13 01:40:58,435 - MidasV1.StockListPetitioner - INFO - Waiting for scanner data (timeout in 15 seconds)...
2024-12-13 01:40:59,213 - MidasV1.InitialChecks - ERROR - [ERROR] id=1001, code=162, msg=Historical Market Data Service error message:Scanner type with code ALL is disabled
2024-12-13 01:41:13,435 - MidasV1.StockListPetitioner - ERROR - Scanner subscription timed out.
2024-12-13 01:41:13,435 - MidasV1.StockListPetitioner - ERROR - Scanner subscription canceled due to timeout.
2024-12-13 01:41:13,436 - MidasV1.Main - INFO - Refined Stock List: []
2024-12-13 01:41:13,436 - MidasV1.Main - INFO - Initial checks and Module 2 completed. Please review the logs and ensure everything is correct.
2024-12-13 01:41:13,439 - MidasV1.InitialChecks - ERROR - [ERROR] id=1001, code=365, msg=No scanner subscription found for ticker id:1001
2024-12-13 01:41:45,958 - MidasV1.Main - ERROR - Interrupt received. Shutting down gracefully...
2024-12-13 01:41:45,958 - MidasV1.InitialChecks - ERROR - [ERROR] Connection to IB Gateway was closed unexpectedly!
2024-12-13 01:41:45,958 - MidasV1.Main - INFO - Disconnected from IB Gateway.
2024-12-13 02:13:25,804 - MidasV1.Main - INFO - Starting MidasV1 Trading Bot...
2024-12-13 02:13:25,804 - MidasV1.InitialChecks - INFO - Checking Operating System...
2024-12-13 02:13:25,804 - MidasV1.InitialChecks - INFO - Operating System Linux is supported.
2024-12-13 02:13:25,804 - MidasV1.InitialChecks - INFO - Checking Dependencies...
2024-12-13 02:13:25,804 - MidasV1.InitialChecks - INFO - All dependencies are satisfied.
2024-12-13 02:13:25,804 - MidasV1.InitialChecks - INFO - Checking System Resources...
2024-12-13 02:13:25,805 - MidasV1.InitialChecks - INFO -
CPU Cores: 12
CPU Clock Speed: 1030.11 MHz
CPU Load Average (1m, 5m, 15m): (0.7734375, 0.642578125, 0.6494140625)
CPU Threads: 12
Total RAM: 15.31 GB
Used RAM: 3.11 GB (27.6%)
Available RAM: 11.09 GB
Total Swap: 0.95 GB
Used Swap: 0.00 GB (0.0%)
2024-12-13 02:13:25,805 - MidasV1.InitialChecks - INFO - Checking Connectivity with IB Gateway...
2024-12-13 02:13:25,805 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 02:13:25,805 - MidasV1.InitialChecks - INFO - IB Gateway Connection Test
2024-12-13 02:13:25,805 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 02:13:25,805 - MidasV1.InitialChecks - INFO - Attempting to connect to IB Gateway at 127.0.0.1:4002...
2024-12-13 02:13:25,809 - MidasV1.InitialChecks - INFO - [INFO] Managed accounts: DUE064818
2024-12-13 02:13:25,851 - MidasV1.InitialChecks - INFO - [INFO] Next valid order ID: 1
2024-12-13 02:13:25,851 - MidasV1.InitialChecks - INFO - [INFO/STATUS] id=-1, code=2104, msg=Market data farm connection is OK:usfarm
2024-12-13 02:13:25,852 - MidasV1.InitialChecks - INFO - [INFO/STATUS] id=-1, code=2107, msg=HMDS data farm connection is inactive but should be available upon demand.ushmds
2024-12-13 02:13:25,852 - MidasV1.InitialChecks - INFO - [INFO/STATUS] id=-1, code=2158, msg=Sec-def data farm connection is OK:secdefnj
2024-12-13 02:13:25,908 - MidasV1.InitialChecks - INFO - [INFO] Connected successfully!
2024-12-13 02:13:25,909 - MidasV1.InitialChecks - INFO - Requesting account summary...
2024-12-13 02:13:25,940 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, BuyingPower = 4014612.28 USD
2024-12-13 02:13:25,940 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, EquityWithLoanValue = 1003653.07 USD
2024-12-13 02:13:25,941 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, NetLiquidation = 1004890.64 USD
2024-12-13 02:13:25,941 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY] ReqId:1, Account:DUE064818, TotalCashValue = 1003653.07 USD
2024-12-13 02:13:25,941 - MidasV1.InitialChecks - INFO - [ACCOUNT SUMMARY END] ReqId: 1
2024-12-13 02:13:30,910 - MidasV1.InitialChecks - INFO - [INFO] Successfully retrieved account summary data.
2024-12-13 02:13:30,910 - MidasV1.InitialChecks - INFO - IB Gateway is connected and ready for upcoming modules.
2024-12-13 02:13:30,911 - MidasV1.InitialChecks - INFO - ────────────────────────────────────────────────────
2024-12-13 02:13:30,911 - MidasV1.InitialChecks - INFO - Test Complete
2024-12-13 02:13:30,911 - MidasV1.Main - ERROR - An unexpected error occurred during initial checks or Module 2.
Traceback (most recent call last):
File "/home/klein/codeWS/Projects/MidasTechnologiesLLC/MidasTechnologies/src/MidasV1/main.py", line 194, in main
connected_client = initial_checks.run_all_checks(skip_checks=args.skip_checks, callback_handlers=[stock_petitioner]) # Pass as callback handler
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/klein/codeWS/Projects/MidasTechnologiesLLC/MidasTechnologies/src/MidasV1/modules/initial_checks.py", line 357, in run_all_checks
connected_client = self.check_connectivity(callback_handlers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/klein/codeWS/Projects/MidasTechnologiesLLC/MidasTechnologies/src/MidasV1/modules/initial_checks.py", line 285, in check_connectivity
app.connect_and_run()
File "/home/klein/codeWS/Projects/MidasTechnologiesLLC/MidasTechnologies/src/MidasV1/modules/initial_checks.py", line 280, in connect_and_run
print(f"{MAGENTA}{BOLD} Test Complete{RESET}")
^^^^^^^
NameError: name 'MAGENTA' is not defined
2024-12-13 02:13:49,362 - MidasV1.Main - ERROR - Interrupt received. Shutting down gracefully...

261
src/MidasV1/main.py Normal file
View File

@@ -0,0 +1,261 @@
# main.py
"""
========================================================================
# README
#
# Program: main.py
#
# Description:
# This script serves as the entry point for the MidasV1 Trading Bot.
# It performs initial system checks, handles configuration loading, sets up logging,
# initializes modules, and manages the overall flow of the application.
#
# Features:
# - Parses command-line arguments for customization.
# - Loads configuration settings from a config file.
# - Sets up colored logging for enhanced visibility.
# - Handles graceful shutdown on interrupt signals (e.g., Ctrl+C).
# - Initializes and runs initial checks and the Stock List Petitioner module.
# - Provides placeholders for integrating additional modules in the future.
#
# Usage:
# Run the script from the command line with optional arguments:
# --no-checks : Run the program without prompting for user confirmation after initial checks.
# --skip-checks : Skip specific initial checks (primarily dependency checks).
# --verbose : Enable verbose and colorful output to the console.
# --version : Print the program version and exit.
#
# Example:
# python main.py --verbose
# python main.py --version
# python main.py --verbose --no-checks
# python main.py --verbose --skip-checks
#
# Coded by: kleinpanic 2024
========================================================================
"""
import argparse
import logging
import sys
import os
import configparser
import signal
import threading
from modules.initial_checks import InitialChecks
from modules.stock_list_petitioner import StockListPetitioner # Import Module 2
class ColoredFormatter(logging.Formatter):
# ANSI escape codes for colors
COLOR_CODES = {
'DEBUG': "\033[94m", # Blue
'INFO': "\033[92m", # Green
'WARNING': "\033[93m", # Yellow
'ERROR': "\033[91m", # Red
'CRITICAL': "\033[95m", # Magenta
}
RESET_CODE = "\033[0m"
def format(self, record):
color = self.COLOR_CODES.get(record.levelname, self.RESET_CODE)
message = super().format(record)
if record.levelname in self.COLOR_CODES:
message = f"{color}{message}{self.RESET_CODE}"
return message
def setup_logging(verbose=False, log_level='INFO'):
"""
Configures logging for the application.
Args:
verbose (bool): If True, set logging level to DEBUG and log to console with colors.
log_level (str): Specific logging level from config.
"""
log_format = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
log_file = os.path.join('logs', 'MidasV1.log')
# Create logs directory if it doesn't exist
os.makedirs('logs', exist_ok=True)
# Determine logging level
level = getattr(logging, log_level.upper(), logging.INFO)
if verbose:
level = logging.DEBUG
logger = logging.getLogger()
logger.setLevel(level)
# Remove existing handlers to prevent duplicate logs
if logger.hasHandlers():
logger.handlers.clear()
# File handler (uncolored, all logs)
file_handler = logging.FileHandler(log_file)
file_handler.setLevel(logging.DEBUG) # Log all levels to file
file_formatter = logging.Formatter(log_format)
file_handler.setFormatter(file_formatter)
logger.addHandler(file_handler)
if verbose:
# Console handler (colored, INFO and above)
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setLevel(logging.INFO)
console_formatter = ColoredFormatter(log_format)
console_handler.setFormatter(console_formatter)
logger.addHandler(console_handler)
def load_config(config_path='config/config.config'):
"""
Loads the configuration file.
Args:
config_path (str): Path to the configuration file.
Returns:
configparser.ConfigParser: Parsed configuration.
"""
config = configparser.ConfigParser()
if not os.path.exists(config_path):
print(f"Configuration file not found at {config_path}. Exiting.")
sys.exit(1)
config.read(config_path)
return config
def parse_arguments():
"""
Parses command-line arguments.
Returns:
argparse.Namespace: Parsed arguments.
"""
parser = argparse.ArgumentParser(description='MidasV1 Trading Bot')
parser.add_argument('--no-checks', action='store_true', help='Run the program without prompting for user confirmation after initial checks')
parser.add_argument('--skip-checks', action='store_true', help='Skip specific initial checks (primarily dependency checks)')
parser.add_argument('--verbose', action='store_true', help='Enable verbose and colorful output to the console')
parser.add_argument('--version', action='store_true', help='Print the program version and exit')
return parser.parse_args()
# Initialize connected_client as global
connected_client = None
def signal_handler(sig, frame):
"""
Handles incoming signals for graceful shutdown.
Args:
sig (int): Signal number.
frame: Current stack frame.
"""
logger = logging.getLogger('MidasV1.Main')
logger.error("Interrupt received. Shutting down gracefully...")
global connected_client
if connected_client:
connected_client.disconnect()
logger.info("Disconnected from IB Gateway.")
sys.exit(0)
def main():
global connected_client # Declare as global to modify the global variable
# Register the signal handler for graceful shutdown
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
# Parse command-line arguments
args = parse_arguments()
# Load configuration
config = load_config()
# Setup logging based on flags and config
log_level = config.get('Logging', 'level', fallback='INFO')
setup_logging(verbose=args.verbose, log_level=log_level)
logger = logging.getLogger('MidasV1.Main')
# Suppress ibapi internal logs from propagating to root logger
ibapi_logger = logging.getLogger('ibapi')
ibapi_logger.setLevel(logging.WARNING) # Suppress DEBUG and INFO
ibapi_logger.propagate = False
# Handle --version flag
if args.version:
version = config.get('General', 'version', fallback='1.0.0')
print(f"MidasV1 Version: {version}")
sys.exit(0)
logger.info("Starting MidasV1 Trading Bot...")
# Initialize and run initial checks if not skipping all checks
if not args.skip_checks:
try:
initial_checks = InitialChecks(config, verbose=args.verbose)
stock_petitioner = StockListPetitioner(config) # Instantiate with only config
connected_client = initial_checks.run_all_checks(skip_checks=args.skip_checks, callback_handlers=[stock_petitioner]) # Pass as callback handler
stock_petitioner.set_client(connected_client) # Set the connected client
refined_stock_list = stock_petitioner.run_module()
logger.info(f"Refined Stock List: {refined_stock_list}") # Log to prevent unused warning
except SystemExit as e:
logger.error("Initial checks failed. Exiting program.")
sys.exit(e.code)
except Exception as e:
logger.exception("An unexpected error occurred during initial checks or Module 2.")
if connected_client:
connected_client.disconnect()
logger.info("Disconnected from IB Gateway.")
sys.exit(1)
else:
logger.warning("Skipping specific initial checks as per the '--skip-checks' flag.")
refined_stock_list = []
# Prompt the user to confirm before proceeding to the next module
if not args.no_checks:
logger.info("Initial checks and Module 2 completed. Please review the logs and ensure everything is correct.")
try:
while True:
user_input = input("Do you want to proceed to the next module? (y/n): ").strip().lower()
if user_input == 'y':
logger.info("User chose to proceed.")
break
elif user_input == 'n':
logger.info("User chose to exit the program.")
if connected_client:
connected_client.disconnect()
logger.info("Disconnected from IB Gateway.")
sys.exit(0)
else:
print("Please enter 'y' or 'n'.")
except KeyboardInterrupt:
logger.error("Interrupt received during user prompt. Shutting down gracefully...")
if connected_client:
connected_client.disconnect()
logger.info("Disconnected from IB Gateway.")
sys.exit(0)
else:
logger.info("Proceeding to the next module without user confirmation as per the '--no-checks' flag.")
# Placeholder for initializing and running other modules (e.g., Module 3)
# Example:
# from modules.module3 import Module3
# module3 = Module3(config, connected_client, refined_stock_list)
# module3.run()
logger.info("MidasV1 Trading Bot is now running.")
# Placeholder for main loop or orchestration logic
try:
while True:
# Implement the main functionality here
# For demonstration, we'll just sleep
threading.Event().wait(1)
except KeyboardInterrupt:
logger.error("Interrupt received. Shutting down gracefully...")
if connected_client:
connected_client.disconnect()
logger.info("Disconnected from IB Gateway.")
sys.exit(0)
if __name__ == '__main__':
main()

1
src/MidasV1/modules/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
backups/

BIN
src/MidasV1/modules/__pycache__/initial_checks.cpython-311.pyc Normal file
View File

Binary file not shown.

BIN
src/MidasV1/modules/__pycache__/stock_list_petitioner.cpython-311.pyc Normal file
View File

Binary file not shown.

312
src/MidasV1/modules/initial_checks.py Normal file
View File

@@ -0,0 +1,312 @@
# modules/initial_checks.py
"""
========================================================================
# README
#
# Module: initial_checks.py
#
# Description:
# This module performs a series of initial system and environment checks
# required before the MidasV1 Trading Bot can operate effectively.
# It verifies the operating system, checks for necessary dependencies,
# assesses system resources, and ensures connectivity with the Interactive Brokers (IB) Gateway.
#
# Features:
# - Checks if the operating system is supported (currently Linux).
# - Verifies that required Python packages (`ibapi`, `psutil`) are installed.
# - Logs detailed system resource information including CPU cores, clock speed,
# load averages, CPU threads, and RAM statistics.
# - Tests connectivity with the IB Gateway by attempting to establish a session
# and retrieve account summaries.
# - Integrates with other modules via callback handlers to facilitate data exchange.
#
# Usage:
# This module is primarily used by `main.py` during the startup phase.
#
# Example:
# from modules.initial_checks import InitialChecks
# config = load_config()
# initial_checks = InitialChecks(config, verbose=True)
# connected_client = initial_checks.run_all_checks(callback_handlers=[stock_petitioner])
#
# Coded by: kleinpanic 2024
========================================================================
"""
import platform
import sys
import logging
import os
import psutil
import threading
import time
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.utils import iswrapper
from ibapi.contract import Contract
from ibapi.account_summary_tags import AccountSummaryTags # Ensure correct import
from ibapi.tag_value import TagValue
SEPARATOR = "────────────────────────────────────────────────────"
class InitialChecks:
def __init__(self, config, verbose=False):
self.config = config
self.verbose = verbose
self.logger = logging.getLogger('MidasV1.InitialChecks')
def check_os(self):
"""
Determines the operating system and verifies if it's supported.
Currently supports only Linux.
"""
self.logger.info("Checking Operating System...")
os_type = platform.system()
if os_type != 'Linux':
message = f"Unsupported Operating System: {os_type}"
self.logger.error(message)
self.logger.warning("Future support for other operating systems is being added.")
sys.exit(1)
success_message = f"Operating System {os_type} is supported."
self.logger.info(success_message)
def check_dependencies(self):
"""
Ensures that all necessary dependencies are installed.
"""
self.logger.info("Checking Dependencies...")
# Check if 'ibapi' and 'psutil' are installed
dependencies = ['ibapi', 'psutil']
missing = []
for dep in dependencies:
try:
__import__(dep)
except ImportError:
missing.append(dep)
if missing:
error_message = f"Missing Dependencies: {', '.join(missing)}"
self.logger.error(error_message)
self.logger.warning("Please install the missing dependencies and try again.")
sys.exit(1)
success_message = "All dependencies are satisfied."
self.logger.info(success_message)
def check_connectivity(self, callback_handlers=[]):
"""
Verifies a secure connection with the IB Gateway.
Args:
callback_handlers (list): List of modules to receive callbacks.
"""
self.logger.info("Checking Connectivity with IB Gateway...")
host = self.config.get('Connectivity', 'host', fallback='127.0.0.1')
port = self.config.getint('Connectivity', 'port', fallback=4002)
client_id = self.config.getint('Connectivity', 'client_id', fallback=0)
# Define wrapper and client for connection test
class TestWrapper(EWrapper):
def __init__(self, logger, callback_handlers):
super().__init__()
self.nextValidOrderId = None
self.connected_flag = False
self.received_account_summary = False
self.logger = logger
self.callback_handlers = callback_handlers
@iswrapper
def error(self, reqId, errorCode, errorString, advancedOrderRejectJson=""):
info_codes = {2104, 2106, 2107, 2158} # Include code 2106 based on your logs
if errorCode in info_codes:
self.logger.info(f"[INFO/STATUS] id={reqId}, code={errorCode}, msg={errorString}")
else:
self.logger.error(f"[ERROR] id={reqId}, code={errorCode}, msg={errorString}")
@iswrapper
def nextValidId(self, orderId: int):
self.logger.info(f"[INFO] Next valid order ID: {orderId}")
self.nextValidOrderId = orderId
self.connected_flag = True
@iswrapper
def accountSummary(self, reqId: int, account: str, tag: str, value: str, currency: str):
self.received_account_summary = True
self.logger.info(f"[ACCOUNT SUMMARY] ReqId:{reqId}, Account:{account}, {tag} = {value} {currency}")
@iswrapper
def accountSummaryEnd(self, reqId: int):
self.logger.info(f"[ACCOUNT SUMMARY END] ReqId: {reqId}")
@iswrapper
def managedAccounts(self, accountsList: str):
self.logger.info(f"[INFO] Managed accounts: {accountsList}")
@iswrapper
def connectionClosed(self):
self.logger.error("[ERROR] Connection to IB Gateway was closed unexpectedly!")
# Notify callback handlers if needed
@iswrapper
def scannerData(self, reqId: int, rank: int, contractDetails: Contract, distance: str,
benchmark: str, projection: str, legsStr: str):
# Dispatch to callback handlers
for handler in self.callback_handlers:
handler.scannerData(reqId, rank, contractDetails, distance, benchmark, projection, legsStr)
@iswrapper
def scannerDataEnd(self, reqId: int):
self.logger.info(f"Scanner data end received for reqId: {reqId}")
for handler in self.callback_handlers:
handler.scannerDataEnd(reqId)
class TestClient(EClient):
def __init__(self, wrapper):
super().__init__(wrapper)
class ConnectionTestApp(TestWrapper, TestClient):
def __init__(self, host: str, port: int, client_id: int, logger, callback_handlers=[]):
TestWrapper.__init__(self, logger, callback_handlers)
TestClient.__init__(self, self)
self.host = host
self.port = port
self.client_id = client_id
def connect_and_run(self):
self.logger.info(SEPARATOR)
self.logger.info(" IB Gateway Connection Test")
self.logger.info(SEPARATOR)
connection_message = f"Attempting to connect to IB Gateway at {self.host}:{self.port}..."
self.logger.info(connection_message)
try:
self.connect(self.host, self.port, self.client_id)
except ConnectionRefusedError:
error_message = "[ERROR] Connection refused. Is IB Gateway running?"
self.logger.error(error_message)
sys.exit(1)
# Start the EClient message processing thread
thread = threading.Thread(target=self.run, daemon=True)
thread.start()
# Wait until connected or timeout
start_time = time.time()
timeout = 5 # seconds
while not self.connected_flag and (time.time() - start_time < timeout):
time.sleep(0.1)
if not self.connected_flag:
error_message = "[ERROR] Connection not established within timeout."
self.logger.error(error_message)
warning_message = "[WARN] No connection. Check Gateway settings and try again."
self.logger.warning(warning_message)
self.disconnect()
sys.exit(1)
success_message = "[INFO] Connected successfully!"
self.logger.info(success_message)
request_message = "Requesting account summary..."
self.logger.info(request_message)
# Request account summary to verify further communication
req_id = 1
self.reqAccountSummary(req_id, "All", "NetLiquidation,TotalCashValue,EquityWithLoanValue,BuyingPower")
# Wait a bit for responses
time.sleep(5)
self.cancelAccountSummary(req_id)
# Check if we received account summary data
if self.received_account_summary:
success_summary = "[INFO] Successfully retrieved account summary data."
self.logger.info(success_summary)
else:
warning_summary = "[WARN] Connected but did not receive account summary data. Is the account funded or available?"
self.logger.warning(warning_summary)
self.logger.info("IB Gateway is connected and ready for upcoming modules.")
self.logger.info(SEPARATOR)
self.logger.info(" Test Complete")
self.logger.info(SEPARATOR)
# Initialize and run the connection test
app = ConnectionTestApp(host, port, client_id, self.logger, callback_handlers)
app.connect_and_run()
# Return the app to keep the connection open
return app
def check_system_resources(self):
"""
Logs system resource information such as CPU cores, clock speed, load averages, CPU threads, and detailed RAM information.
The system information is colored gold for better visibility.
"""
self.logger.info("Checking System Resources...")
# Gather system information
cpu_cores = psutil.cpu_count(logical=True)
cpu_freq = psutil.cpu_freq()
cpu_load_avg = psutil.getloadavg()
cpu_threads = psutil.cpu_count(logical=True)
ram = psutil.virtual_memory()
swap = psutil.swap_memory()
total_ram_gb = ram.total / (1024 ** 3)
used_ram_gb = ram.used / (1024 ** 3)
available_ram_gb = ram.available / (1024 ** 3)
ram_percent = ram.percent
swap_total_gb = swap.total / (1024 ** 3)
swap_used_gb = swap.used / (1024 ** 3)
swap_percent = swap.percent
# Construct the resource information string
resource_info = (
f"CPU Cores: {cpu_cores}\n"
f"CPU Clock Speed: {cpu_freq.current:.2f} MHz\n"
f"CPU Load Average (1m, 5m, 15m): {cpu_load_avg}\n"
f"CPU Threads: {cpu_threads}\n"
f"Total RAM: {total_ram_gb:.2f} GB\n"
f"Used RAM: {used_ram_gb:.2f} GB ({ram_percent}%)\n"
f"Available RAM: {available_ram_gb:.2f} GB\n"
f"Total Swap: {swap_total_gb:.2f} GB\n"
f"Used Swap: {swap_used_gb:.2f} GB ({swap_percent}%)"
)
# ANSI escape code for gold (approximated by yellow)
gold_color = "\033[93m" # Bright Yellow as gold approximation
reset_color = "\033[0m"
# Combine the newline and colored resource information
colored_resource_info = f"\n{gold_color}{resource_info}{reset_color}"
# Log the colored resource information
self.logger.info(colored_resource_info)
def run_all_checks(self, skip_checks=False, callback_handlers=[]):
"""
Executes all initial checks in the required sequence.
Args:
skip_checks (bool): If True, skips specific checks like dependency checks.
callback_handlers (list): List of modules to receive callbacks.
Returns:
connected_client: The connected ibapi client to be used by other modules.
"""
self.check_os()
if not skip_checks:
self.check_dependencies()
else:
warning_message = "Skipping dependency checks as per the '--skip-checks' flag."
self.logger.warning(warning_message)
self.check_system_resources()
connected_client = self.check_connectivity(callback_handlers)
success_message = "All initial checks passed successfully."
self.logger.info(success_message)
return connected_client

444
src/MidasV1/modules/stock_list_petitioner.py Normal file
View File

@@ -0,0 +1,444 @@
# modules/stock_list_petitioner.py
"""
========================================================================
# README
#
# Module: stock_list_petitioner.py
#
# Description:
# This module handles the process of requesting and refining a list of stock symbols
# based on predefined criteria. It interacts with the Interactive Brokers (IB) API
# to perform scanner subscriptions, receive market data, and apply filters to
# generate a refined list of stocks suitable for trading strategies.
#
# Features:
# - Initiates scanner subscriptions to retrieve stock data based on criteria such as
# search volume and net change.
# - Receives and processes scanner data asynchronously using IB API callbacks.
# - Refines the received stock list by applying additional criteria like share price,
# availability of option contracts, and volatility index.
# - Caches the refined stock list for use by subsequent modules.
# - Provides detailed logging and colored console outputs for better traceability and user feedback.
#
# Usage:
# This module is instantiated and used by `main.py` after initial checks are passed.
#
# Example:
# from modules.stock_list_petitioner import StockListPetitioner
# stock_petitioner = StockListPetitioner(config)
# connected_client = initial_checks.run_all_checks(callback_handlers=[stock_petitioner])
# refined_stock_list = stock_petitioner.run_module()
#
# Coded by: kleinpanic 2024
========================================================================
"""
import logging
import threading
import tempfile
import json
import time
import os # Added import for os
from ibapi.contract import Contract
from ibapi.scanner import ScannerSubscription
from ibapi.ticktype import TickTypeEnum
from ibapi.utils import iswrapper
from ibapi.tag_value import TagValue
class StockListPetitioner:
def __init__(self, config):
self.logger = logging.getLogger('MidasV1.StockListPetitioner')
self.config = config
self.connected_client = None # To be set later
# Scanner results
self.scanner_data = []
self.scanner_finished = False
self.lock = threading.Lock()
# Event to signal when scanner data is received
self.scanner_event = threading.Event()
def set_client(self, connected_client):
"""
Sets the connected_client after initial checks.
"""
self.connected_client = connected_client
def run_module(self):
"""
Executes the scanner subscription and refines the stock list.
"""
if not self.connected_client:
self.logger.error("Connected client is not set. Cannot proceed with scanner subscription.")
return []
self.logger.info("Starting Module 2: IBJTS List Petitioner...")
# Load scanner criteria from config
search_volume = self.config.getint('Module2', 'default_search_volume', fallback=10000)
net_change = self.config.getfloat('Module2', 'default_net_change', fallback=0.0)
percent_change = self.config.getfloat('Module2', 'default_percent_change', fallback=0.0)
# Display and log criteria
criteria_message = (
f"Loaded Scanner Criteria:\n"
f" - Search Volume: {search_volume}\n"
f" - Net Change: {net_change}\n"
f" - Percent Change: {percent_change}"
)
self.logger.info(criteria_message)
print("\033[94m" + criteria_message + "\033[0m") # Blue text for criteria
# Define the scanner subscription
subscription = ScannerSubscription()
subscription.instrument = "STK"
subscription.locationCode = "STK.US.MAJOR" # Broad location code to include major US stocks
subscription.scanCode = "ALL" # Broader scan code to include all stocks
subscription.aboveVolume = search_volume
# subscription.netChange = net_change # Removed for compatibility with "ALL"
# subscription.percentChange = percent_change # Removed for compatibility with "ALL"
# Inform user about the API call
api_call_message = "Initiating scanner subscription with the above criteria..."
self.logger.info(api_call_message)
print("\033[92m" + api_call_message + "\033[0m") # Green text for API call info
# Optionally, implement retries
MAX_RETRIES = 2
for attempt in range(1, MAX_RETRIES + 1):
try:
self.connected_client.reqScannerSubscription(
reqId=1001,
subscription=subscription,
scannerSubscriptionOptions=[], # Can be extended based on config
scannerSubscriptionFilterOptions=[] # Can be extended based on config
)
self.logger.info(f"Scanner subscription requested successfully on attempt {attempt}.")
print(f"\033[92mScanner subscription requested successfully on attempt {attempt}.\033[0m") # Green text
break
except Exception as e:
self.logger.error(f"Attempt {attempt}: Error in reqScannerSubscription: {e}")
print(f"\033[91mAttempt {attempt}: Error in reqScannerSubscription: {e}\033[0m") # Red text
if attempt == MAX_RETRIES:
return []
time.sleep(2 ** attempt) # Exponential backoff
# Wait for scanner data or timeout
scanner_timeout = 15 # seconds
self.logger.info(f"Waiting for scanner data (timeout in {scanner_timeout} seconds)...")
print(f"\033[93mWaiting for scanner data (timeout in {scanner_timeout} seconds)...\033[0m") # Yellow text
scanner_completed = self.scanner_event.wait(timeout=scanner_timeout)
if not scanner_completed:
self.logger.error("Scanner subscription timed out.")
try:
self.connected_client.cancelScannerSubscription(1001)
self.logger.error("Scanner subscription canceled due to timeout.")
print("\033[91mScanner subscription timed out and was canceled.\033[0m") # Red text
except Exception as e:
self.logger.error(f"Error canceling scanner subscription: {e}")
print(f"\033[91mError canceling scanner subscription: {e}\033[0m") # Red text
return []
self.logger.info("Scanner data received. Proceeding to refine the stock list.")
print("\033[92mScanner data received. Proceeding to refine the stock list.\033[0m") # Green text
# Log the number of scanner data entries received
self.logger.debug(f"Total scanner data received: {len(self.scanner_data)}")
print(f"\033[94mTotal scanner data received: {len(self.scanner_data)}\033[0m") # Blue text
for stock in self.scanner_data:
self.logger.debug(f"Stock: {stock['symbol']}, Volume: {stock['distance']}")
# Optionally, print detailed scanner data for debugging
# print(f"\033[94mStock: {stock['symbol']}, Volume: {stock['distance']}\033[0m") # Blue text
# Process and refine the scanner data
refined_list = self.refine_stock_list()
# Cache the refined list
self.cache_refined_list(refined_list)
# Print the refined list to the user
self.print_refined_list(refined_list)
if not refined_list:
self.logger.error("No stocks meet the specified criteria after refinement.")
print("\033[91mNo stocks meet the specified criteria after refinement.\033[0m\n") # Red text
# Disconnect the client to prevent further logging
try:
self.connected_client.disconnect()
self.logger.info("Disconnected from IB Gateway after Module 2.")
print("\033[92mDisconnected from IB Gateway after Module 2.\033[0m") # Green text
except Exception as e:
self.logger.error(f"Error disconnecting from IB Gateway: {e}")
print(f"\033[91mError disconnecting from IB Gateway: {e}\033[0m") # Red text
self.logger.info("Module 2: IBJTS List Petitioner completed successfully.")
print("\033[92mModule 2: IBJTS List Petitioner completed successfully.\033[0m") # Green text
return refined_list
@iswrapper
def scannerData(self, reqId: int, rank: int, contractDetails, distance: str,
benchmark: str, projection: str, legsStr: str):
"""
Receives scanner data.
"""
with self.lock:
self.scanner_data.append({
'rank': rank,
'symbol': contractDetails.contract.symbol,
'sectype': contractDetails.contract.secType,
'exchange': contractDetails.contract.exchange,
'currency': contractDetails.contract.currency,
'distance': distance,
'benchmark': benchmark,
'projection': projection,
'legsStr': legsStr
})
self.logger.debug(f"Received scanner data: {self.scanner_data[-1]}")
@iswrapper
def scannerDataEnd(self, reqId: int):
"""
Indicates the end of scanner data.
"""
self.logger.info(f"Scanner data end received for reqId: {reqId}")
with self.lock:
self.scanner_finished = True
self.scanner_event.set()
def refine_stock_list(self):
"""
Refines the scanner data based on additional criteria.
Returns:
list: Refined list of stocks.
"""
self.logger.info("Refining the stock list based on criteria...")
print("\033[93mRefining the stock list based on criteria...\033[0m") # Yellow text
refined_list = []
for stock in self.scanner_data:
symbol = stock['symbol']
self.logger.debug(f"Processing stock: {symbol}")
print(f"\033[94mProcessing stock: {symbol}\033[0m") # Blue text
# Fetch additional data for each stock
share_price = self.get_share_price(symbol)
if share_price is None:
self.logger.debug(f"Skipping {symbol}: Unable to retrieve share price.")
print(f"\033[91mSkipping {symbol}: Unable to retrieve share price.\033[0m") # Red text
continue # Skip if unable to fetch share price
if share_price > self.config.getfloat('Module2', 'default_refinement_share_price', fallback=15.0):
self.logger.debug(f"Excluding {symbol}: Share price ${share_price} exceeds threshold.")
print(f"\033[91mExcluding {symbol}: Share price ${share_price} exceeds threshold.\033[0m") # Red text
continue # Remove stocks above the share price threshold
if not self.has_option_contracts(symbol):
self.logger.debug(f"Excluding {symbol}: No option contracts available.")
print(f"\033[91mExcluding {symbol}: No option contracts available.\033[0m") # Red text
continue # Remove stocks without option contracts
volatility_index = self.get_volatility_index(symbol)
if volatility_index is None:
self.logger.debug(f"Skipping {symbol}: Unable to retrieve volatility index.")
print(f"\033[91mSkipping {symbol}: Unable to retrieve volatility index.\033[0m") # Red text
continue # Skip if unable to fetch volatility index
if volatility_index > self.config.getfloat('Module2', 'default_volatility_threshold', fallback=30.0):
self.logger.debug(f"Excluding {symbol}: Volatility index {volatility_index}% exceeds threshold.")
print(f"\033[91mExcluding {symbol}: Volatility index {volatility_index}% exceeds threshold.\033[0m") # Red text
continue # Remove stocks above the volatility threshold
# Append to refined list if all criteria are met
refined_list.append({
'symbol': symbol,
'share_price': share_price,
'volatility_index': volatility_index
})
self.logger.debug(f"Including {symbol}: Meets all criteria.")
print(f"\033[92mIncluding {symbol}: Meets all criteria.\033[0m") # Green text
# Conditional refinement based on config
conditional_refinement = self.config.getboolean('Module2', 'conditional_refinement_enabled', fallback=False)
if conditional_refinement:
max_list_size = self.config.getint('Module2', 'max_refined_list_size', fallback=100)
if len(refined_list) > max_list_size:
refined_list = refined_list[:max_list_size]
self.logger.info(f"List truncated to {max_list_size} items based on conditional refinement.")
print(f"\033[93mList truncated to {max_list_size} items based on conditional refinement.\033[0m") # Yellow text
self.logger.info(f"Refined list contains {len(refined_list)} stocks after applying all filters.")
print(f"\033[94mRefined list contains {len(refined_list)} stocks after applying all filters.\033[0m") # Blue text
return refined_list
def get_share_price(self, symbol):
"""
Retrieves the current share price for a given symbol.
Args:
symbol (str): Stock symbol.
Returns:
float: Current share price or None if unavailable.
"""
contract = Contract()
contract.symbol = symbol
contract.secType = "STK"
contract.exchange = "SMART"
contract.currency = "USD"
price = None
price_event = threading.Event()
def tickPrice_override(reqId, tickType, price_value, attrib):
nonlocal price
if tickType == TickTypeEnum.LAST:
price = price_value
price_event.set()
# Temporarily override the tickPrice callback
original_tickPrice = self.connected_client.wrapper.tickPrice
self.connected_client.wrapper.tickPrice = tickPrice_override
# Request market data
try:
self.connected_client.reqMktData(2001, contract, "", False, False, [])
self.logger.debug(f"Requested market data for {symbol}.")
print(f"\033[94mRequested market data for {symbol}.\033[0m") # Blue text
except Exception as e:
self.logger.error(f"Error requesting market data for {symbol}: {e}")
print(f"\033[91mError requesting market data for {symbol}: {e}\033[0m") # Red text
self.connected_client.wrapper.tickPrice = original_tickPrice
return None
# Wait for the price to be received or timeout
if not price_event.wait(timeout=5):
self.logger.warning(f"Timeout while waiting for share price of {symbol}.")
print(f"\033[91mTimeout while waiting for share price of {symbol}.\033[0m") # Red text
else:
self.logger.debug(f"Share price for {symbol}: ${price}")
print(f"\033[92mShare price for {symbol}: ${price}\033[0m") # Green text
# Restore the original tickPrice callback
self.connected_client.wrapper.tickPrice = original_tickPrice
if price is not None:
return price
else:
self.logger.warning(f"Unable to retrieve share price for {symbol}.")
print(f"\033[91mUnable to retrieve share price for {symbol}.\033[0m") # Red text
return None
def has_option_contracts(self, symbol):
"""
Checks if option contracts are available for a given symbol.
Args:
symbol (str): Stock symbol.
Returns:
bool: True if options are available, False otherwise.
"""
contract = Contract()
contract.symbol = symbol
contract.secType = "OPT"
contract.exchange = "SMART"
contract.currency = "USD"
has_options = False
option_event = threading.Event()
def contractDetails_override(reqId, contractDetails):
nonlocal has_options
if contractDetails.contract.symbol == symbol:
has_options = True
option_event.set()
# Temporarily override the contractDetails callback
original_contractDetails = self.connected_client.wrapper.contractDetails
self.connected_client.wrapper.contractDetails = contractDetails_override
# Request contract details
try:
self.connected_client.reqContractDetails(3001, contract)
self.logger.debug(f"Requested contract details for options of {symbol}.")
print(f"\033[94mRequested contract details for options of {symbol}.\033[0m") # Blue text
except Exception as e:
self.logger.error(f"Error requesting contract details for {symbol}: {e}")
print(f"\033[91mError requesting contract details for {symbol}: {e}\033[0m") # Red text
self.connected_client.wrapper.contractDetails = original_contractDetails
return False
# Wait for the callback or timeout
if not option_event.wait(timeout=5):
self.logger.warning(f"Timeout while checking options for {symbol}.")
print(f"\033[91mTimeout while checking options for {symbol}.\033[0m") # Red text
else:
self.logger.debug(f"Options availability for {symbol}: {has_options}")
print(f"\033[92mOptions availability for {symbol}: {has_options}\033[0m") # Green text
# Restore the original contractDetails callback
self.connected_client.wrapper.contractDetails = original_contractDetails
return has_options
def get_volatility_index(self, symbol):
"""
Retrieves the volatility index for a given symbol.
This is a placeholder function. Implement actual volatility retrieval as needed.
Args:
symbol (str): Stock symbol.
Returns:
float: Volatility index or None if unavailable.
"""
# Placeholder implementation
# Replace this with actual implementation, e.g., using an external API or IB's data
mock_volatility = 25.0 # Example value
self.logger.debug(f"Volatility index for {symbol}: {mock_volatility}%")
print(f"\033[94mVolatility index for {symbol}: {mock_volatility}%\033[0m") # Blue text
return mock_volatility
def cache_refined_list(self, refined_list):
"""
Caches the refined stock list in a temporary file for transfer between modules.
Args:
refined_list (list): Refined list of stocks.
"""
try:
timestamp = int(time.time())
cache_path = os.path.join(tempfile.gettempdir(), f"refined_stock_list_{timestamp}.json")
with open(cache_path, 'w') as tmp_file:
json.dump(refined_list, tmp_file)
self.logger.info(f"Refined stock list cached at {cache_path}")
print(f"\033[92mRefined stock list cached at {cache_path}\033[0m") # Green text
except Exception as e:
self.logger.error(f"Failed to cache refined stock list: {e}")
print(f"\033[91mFailed to cache refined stock list: {e}\033[0m") # Red text
def print_refined_list(self, refined_list):
"""
Prints the refined stock list to the user.
Args:
refined_list (list): Refined list of stocks.
"""
if not refined_list:
self.logger.error("No stocks meet the specified criteria after refinement.")
print("\033[91mNo stocks meet the specified criteria after refinement.\033[0m\n") # Red text
return
self.logger.info("Refined Stock List:")
print("\n\033[92mRefined Stock List:\033[0m") # Green text
print("--------------------")
for stock in refined_list:
print(f"Symbol: {stock['symbol']}, Share Price: ${stock['share_price']:.2f}, Volatility Index: {stock['volatility_index']}%")
print("--------------------\n")

2
src/MidasV1/requirements.txt Normal file
View File

@@ -0,0 +1,2 @@
ibapi==10.32.1
psutil==6.1.0

183
src/MidasV1/tests/test_connection.py Normal file
View File

@@ -0,0 +1,183 @@
# test_connection.py
"""
========================================================================
# README
#
# Program: test_connection.py
#
# Description:
# This script tests the connectivity between the MidasV1 Trading Bot and the
# Interactive Brokers (IB) Gateway. It attempts to establish a connection,
# request account summaries, and verify that data is being received correctly.
# The script provides colored console outputs to indicate the status of each
# operation, enhancing user feedback and readability.
#
# Features:
# - Checks for the presence of the `ibapi` Python package and prompts installation if missing.
# - Attempts to connect to the IB Gateway using specified host, port, and client ID.
# - Requests account summary data to verify successful communication.
# - Implements colored console outputs using ANSI escape codes:
# - Green for successful operations.
# - Red for errors.
# - Yellow for warnings and informational messages.
# - Blue and Magenta for decorative separators and headers.
# - Handles graceful shutdown on interrupts and errors.
#
# Usage:
# Run the script from the command line:
# python test_connection.py
#
# The script will display messages indicating the progress and outcome of each step.
#
# Coded by: kleinpanic 2024
========================================================================
"""
import sys
import time
import threading
# Check for ibapi dependency
try:
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.utils import iswrapper
except ImportError:
print("┌───────────────────────────────────────────────────┐")
print("│ IB API Python Not Found! │")
print("└───────────────────────────────────────────────────┘")
print("\nThe 'ibapi' package is not installed. Please install it by running:")
print(" pip install ibapi")
sys.exit(1)
# ANSI color codes
GREEN = "\033[92m"
RED = "\033[91m"
YELLOW = "\033[93m"
BLUE = "\033[94m"
MAGENTA = "\033[95m"
BOLD = "\033[1m"
RESET = "\033[0m"
SEPARATOR = MAGENTA + "────────────────────────────────────────────────────" + RESET
class TestWrapper(EWrapper):
def __init__(self):
super().__init__()
self.nextValidOrderId = None
self.connected_flag = False
self.received_account_summary = False
@iswrapper
def error(self, reqId, errorCode, errorString, advancedOrderRejectJson=""):
# Distinguish between known "info" messages and real errors
# For example, errorCode=2104,2107,2158 are typically info:
info_codes = {2104, 2107, 2158}
if errorCode in info_codes:
# Print these in a different color to indicate they are not severe
print(f"{YELLOW}[INFO/STATUS] id={reqId}, code={errorCode}, msg={errorString}{RESET}")
else:
# True errors
print(f"{RED}[ERROR] id={reqId}, code={errorCode}, msg={errorString}{RESET}")
@iswrapper
def nextValidId(self, orderId: int):
print(f"{GREEN}[INFO] Next valid order ID: {orderId}{RESET}")
self.nextValidOrderId = orderId
self.connected_flag = True
@iswrapper
def accountSummary(self, reqId: int, account: str, tag: str, value: str, currency: str):
self.received_account_summary = True
print(f"{GREEN}[ACCOUNT SUMMARY] ReqId:{reqId}, Account:{account}, {tag} = {value} {currency}{RESET}")
@iswrapper
def accountSummaryEnd(self, reqId: int):
print(f"{GREEN}[ACCOUNT SUMMARY END] ReqId: {reqId}{RESET}")
@iswrapper
def managedAccounts(self, accountsList: str):
print(f"{GREEN}[INFO] Managed accounts: {accountsList}{RESET}")
class TestClient(EClient):
def __init__(self, wrapper):
super().__init__(wrapper)
class ConnectionTestApp(TestWrapper, TestClient):
def __init__(self, host: str, port: int, client_id: int):
TestWrapper.__init__(self)
TestClient.__init__(self, self)
self.host = host
self.port = port
self.client_id = client_id
def connect_and_run(self):
print(SEPARATOR)
print(f"{BOLD}{BLUE} IB Gateway Connection Test{RESET}")
print(SEPARATOR)
print(f"{BLUE}Attempting to connect to IB Gateway at {self.host}:{self.port}...{RESET}")
# Attempt connection with error handling
try:
self.connect(self.host, self.port, self.client_id)
except ConnectionRefusedError:
print(f"{RED}[ERROR] Connection refused. Is IB Gateway running?{RESET}")
return
# Start the EClient message processing thread
thread = threading.Thread(target=self.run, daemon=True)
thread.start()
# Wait until connected or timeout
start_time = time.time()
timeout = 5 # seconds
while not self.connected_flag and (time.time() - start_time < timeout):
time.sleep(0.1)
if not self.connected_flag:
print(f"{RED}[ERROR] Connection not established within timeout.{RESET}")
self.disconnect()
print(f"{YELLOW}[WARN] No connection. Check Gateway settings and try again.{RESET}")
return
print(f"{GREEN}[INFO] Connected successfully!{RESET}")
print(f"{BLUE}Requesting account summary...{RESET}")
# Request account summary to verify further communication
req_id = 1
self.reqAccountSummary(req_id, "All", "NetLiquidation,TotalCashValue,EquityWithLoanValue,BuyingPower")
# Wait a bit for responses
time.sleep(5)
self.cancelAccountSummary(req_id)
self.disconnect()
# Check if we received account summary data
if self.received_account_summary:
print(f"{GREEN}[INFO] Successfully retrieved account summary data.{RESET}")
else:
print(f"{YELLOW}[WARN] Connected but did not receive account summary data. Is the account funded or available?{RESET}")
print(f"{GREEN}[INFO] Disconnected successfully.{RESET}")
print(SEPARATOR)
print(f"{BOLD}{BLUE} Test Complete{RESET}")
print(SEPARATOR)
if __name__ == "__main__":
host = "127.0.0.1"
port = 4002 # Paper trading port
client_id = 0 # Choose a unique client ID
# Check Python version
if sys.version_info < (3, 6):
print(f"{RED}[ERROR] Python 3.6+ is required for this script.{RESET}")
sys.exit(1)
app = ConnectionTestApp(host, port, client_id)
app.connect_and_run()

435
src/MidasV1/tests/test_stock_retriever.log Normal file
View File

@@ -0,0 +1,435 @@
2024-12-13 02:01:32,318 - DEBUG - 139881771299920 connState: None -> 0
2024-12-13 02:01:32,318 - DEBUG - 139881771299920 connState: None -> 0
2024-12-13 02:01:32,319 - DEBUG - Connecting to 127.0.0.1:4002 w/ id:1
2024-12-13 02:01:32,319 - DEBUG - 139881771299920 connState: 0 -> 1
2024-12-13 02:01:32,319 - DEBUG - msg b'\x00\x00\x00\tv100..193'
2024-12-13 02:01:32,319 - DEBUG - REQUEST b'API\x00\x00\x00\x00\tv100..193'
2024-12-13 02:01:32,319 - DEBUG - acquiring lock
2024-12-13 02:01:32,319 - DEBUG - acquired lock
2024-12-13 02:01:32,319 - DEBUG - releasing lock
2024-12-13 02:01:32,319 - DEBUG - release lock
2024-12-13 02:01:32,319 - DEBUG - sendMsg: sent: 17
2024-12-13 02:01:32,320 - DEBUG - no fields
2024-12-13 02:01:32,321 - DEBUG - len 30 raw:b'\x00\x00\x00\x1a187\x0020241213 02:01:31 EST\x00'|
2024-12-13 02:01:32,322 - DEBUG - ANSWER b'\x00\x00\x00\x1a187\x0020241213 02:01:31 EST\x00'
2024-12-13 02:01:32,322 - DEBUG - read_msg: size: 26
2024-12-13 02:01:32,322 - DEBUG - size:26 msg:b'187\x0020241213 02:01:31 EST\x00' rest:b''|
2024-12-13 02:01:32,322 - DEBUG - fields (b'187', b'20241213 02:01:31 EST')
2024-12-13 02:01:32,322 - DEBUG - ANSWER Version:187 time:b'20241213 02:01:31 EST'
2024-12-13 02:01:32,322 - DEBUG - 139881771299920 connState: 1 -> 2
2024-12-13 02:01:32,322 - DEBUG - EReader thread started
2024-12-13 02:01:32,322 - INFO - sent startApi
2024-12-13 02:01:32,322 - INFO - REQUEST startApi {}
2024-12-13 02:01:32,322 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,322 - INFO - SENDING startApi b'\x00\x00\x00\x0871\x002\x001\x00\x00'
2024-12-13 02:01:32,322 - DEBUG - acquiring lock
2024-12-13 02:01:32,322 - DEBUG - acquired lock
2024-12-13 02:01:32,322 - DEBUG - releasing lock
2024-12-13 02:01:32,322 - DEBUG - release lock
2024-12-13 02:01:32,322 - DEBUG - sendMsg: sent: 12
2024-12-13 02:01:32,322 - INFO - ANSWER connectAck {}
2024-12-13 02:01:32,322 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,323 - DEBUG - len 19 raw:b'\x00\x00\x00\x0f15\x001\x00DUE064818\x00'|
2024-12-13 02:01:32,323 - DEBUG - reader loop, recvd size 19
2024-12-13 02:01:32,323 - DEBUG - read_msg: size: 15
2024-12-13 02:01:32,323 - DEBUG - size:15 msg.size:15 msg:|b''| buf:||
2024-12-13 02:01:32,323 - DEBUG - fields (b'15', b'1', b'DUE064818')
2024-12-13 02:01:32,323 - DEBUG - In interpret(), handleInfo: wrap:<function EWrapper.managedAccounts at 0x7f38c36bd3a0> meth:None prms:OrderedDict([('self', <Parameter "self">), ('accountsList', <Parameter "accountsList: str">)])
2024-12-13 02:01:32,323 - DEBUG - field b'DUE064818'
2024-12-13 02:01:32,324 - DEBUG - arg DUE064818 type <class 'str'>
2024-12-13 02:01:32,324 - DEBUG - calling <bound method EWrapper.managedAccounts of <__main__.TestStockRetriever object at 0x7f38c349f450>> with <__main__.TestStockRetriever object at 0x7f38c349f450> ['DUE064818']
2024-12-13 02:01:32,324 - INFO - ANSWER managedAccounts {'accountsList': 'DUE064818'}
2024-12-13 02:01:32,324 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,324 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:32,324 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,366 - DEBUG - len 227 raw:b'\x00\x00\x00\x069\x001\x001\x00\x00\x00\x0064\x002\x00-1\x002104\x00Market data farm connection is OK:usfarm\x00\x00\x00\x00\x00^4\x002\x00-1\x002107\x00HMDS data farm connection is inactive but should be available upon demand.ushmds\x00\x00\x00\x00\x0094\x002\x00-1\x002158\x00Sec-def data farm connection is OK:secdefnj\x00\x00'|
2024-12-13 02:01:32,366 - DEBUG - reader loop, recvd size 227
2024-12-13 02:01:32,366 - DEBUG - read_msg: size: 6
2024-12-13 02:01:32,366 - DEBUG - size:6 msg.size:6 msg:|b'\x00\x00\x0064\x002\x00-1\x002104\x00Market data farm connection is OK:usfarm\x00\x00\x00\x00\x00^4\x002\x00-1\x002107\x00HMDS data farm connection is inactive but should be available upon demand.ushmds\x00\x00\x00\x00\x0094\x002\x00-1\x002158\x00Sec-def data farm connection is OK:secdefnj\x00\x00'| buf:||
2024-12-13 02:01:32,366 - DEBUG - read_msg: size: 54
2024-12-13 02:01:32,366 - DEBUG - size:54 msg.size:54 msg:|b'\x00\x00\x00^4\x002\x00-1\x002107\x00HMDS data farm connection is inactive but should be available upon demand.ushmds\x00\x00\x00\x00\x0094\x002\x00-1\x002158\x00Sec-def data farm connection is OK:secdefnj\x00\x00'| buf:||
2024-12-13 02:01:32,366 - DEBUG - read_msg: size: 94
2024-12-13 02:01:32,366 - DEBUG - size:94 msg.size:94 msg:|b'\x00\x00\x0094\x002\x00-1\x002158\x00Sec-def data farm connection is OK:secdefnj\x00\x00'| buf:||
2024-12-13 02:01:32,366 - DEBUG - read_msg: size: 57
2024-12-13 02:01:32,367 - DEBUG - size:57 msg.size:57 msg:|b''| buf:||
2024-12-13 02:01:32,367 - DEBUG - fields (b'9', b'1', b'1')
2024-12-13 02:01:32,367 - DEBUG - In interpret(), handleInfo: wrap:<function EWrapper.nextValidId at 0x7f38c36bce00> meth:None prms:OrderedDict([('self', <Parameter "self">), ('orderId', <Parameter "orderId: int">)])
2024-12-13 02:01:32,367 - DEBUG - field b'1'
2024-12-13 02:01:32,367 - DEBUG - arg 1 type <class 'int'>
2024-12-13 02:01:32,367 - DEBUG - calling <bound method TestStockRetriever.nextValidId of <__main__.TestStockRetriever object at 0x7f38c349f450>> with <__main__.TestStockRetriever object at 0x7f38c349f450> [1]
2024-12-13 02:01:32,367 - INFO - NextValidId received: 1
2024-12-13 02:01:32,367 - INFO - Requesting market data for symbols...
2024-12-13 02:01:32,367 - INFO - REQUEST reqMktData {'reqId': 1001, 'contract': 139881771406224: 0,AAPL,STK,,,0,,,SMART,,USD,,,False,,,,combo:, 'genericTickList': '', 'snapshot': False, 'regulatorySnapshot': False, 'mktDataOptions': []}
2024-12-13 02:01:32,367 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,367 - INFO - SENDING reqMktData b'\x00\x00\x0011\x0011\x001001\x000\x00AAPL\x00STK\x00\x000.0\x00\x00\x00SMART\x00\x00USD\x00\x00\x000\x00\x000\x000\x00\x00'
2024-12-13 02:01:32,367 - DEBUG - acquiring lock
2024-12-13 02:01:32,367 - DEBUG - acquired lock
2024-12-13 02:01:32,367 - DEBUG - releasing lock
2024-12-13 02:01:32,367 - DEBUG - release lock
2024-12-13 02:01:32,367 - DEBUG - sendMsg: sent: 53
2024-12-13 02:01:32,367 - DEBUG - Requested market data for AAPL with ReqId: 1001
2024-12-13 02:01:32,367 - INFO - REQUEST reqMktData {'reqId': 1002, 'contract': 139881771409296: 0,MSFT,STK,,,0,,,SMART,,USD,,,False,,,,combo:, 'genericTickList': '', 'snapshot': False, 'regulatorySnapshot': False, 'mktDataOptions': []}
2024-12-13 02:01:32,367 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,367 - INFO - SENDING reqMktData b'\x00\x00\x0011\x0011\x001002\x000\x00MSFT\x00STK\x00\x000.0\x00\x00\x00SMART\x00\x00USD\x00\x00\x000\x00\x000\x000\x00\x00'
2024-12-13 02:01:32,367 - DEBUG - acquiring lock
2024-12-13 02:01:32,367 - DEBUG - acquired lock
2024-12-13 02:01:32,367 - DEBUG - releasing lock
2024-12-13 02:01:32,367 - DEBUG - release lock
2024-12-13 02:01:32,367 - DEBUG - sendMsg: sent: 53
2024-12-13 02:01:32,368 - DEBUG - Requested market data for MSFT with ReqId: 1002
2024-12-13 02:01:32,368 - INFO - REQUEST reqMktData {'reqId': 1003, 'contract': 139881771409424: 0,GOOGL,STK,,,0,,,SMART,,USD,,,False,,,,combo:, 'genericTickList': '', 'snapshot': False, 'regulatorySnapshot': False, 'mktDataOptions': []}
2024-12-13 02:01:32,368 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,368 - INFO - SENDING reqMktData b'\x00\x00\x0021\x0011\x001003\x000\x00GOOGL\x00STK\x00\x000.0\x00\x00\x00SMART\x00\x00USD\x00\x00\x000\x00\x000\x000\x00\x00'
2024-12-13 02:01:32,368 - DEBUG - acquiring lock
2024-12-13 02:01:32,368 - DEBUG - acquired lock
2024-12-13 02:01:32,368 - DEBUG - releasing lock
2024-12-13 02:01:32,368 - DEBUG - release lock
2024-12-13 02:01:32,368 - DEBUG - sendMsg: sent: 54
2024-12-13 02:01:32,368 - DEBUG - Requested market data for GOOGL with ReqId: 1003
2024-12-13 02:01:32,368 - INFO - REQUEST reqMktData {'reqId': 1004, 'contract': 139881771409360: 0,AMZN,STK,,,0,,,SMART,,USD,,,False,,,,combo:, 'genericTickList': '', 'snapshot': False, 'regulatorySnapshot': False, 'mktDataOptions': []}
2024-12-13 02:01:32,368 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,368 - INFO - SENDING reqMktData b'\x00\x00\x0011\x0011\x001004\x000\x00AMZN\x00STK\x00\x000.0\x00\x00\x00SMART\x00\x00USD\x00\x00\x000\x00\x000\x000\x00\x00'
2024-12-13 02:01:32,368 - DEBUG - acquiring lock
2024-12-13 02:01:32,368 - DEBUG - acquired lock
2024-12-13 02:01:32,368 - DEBUG - releasing lock
2024-12-13 02:01:32,368 - DEBUG - release lock
2024-12-13 02:01:32,368 - DEBUG - sendMsg: sent: 53
2024-12-13 02:01:32,368 - DEBUG - Requested market data for AMZN with ReqId: 1004
2024-12-13 02:01:32,368 - INFO - REQUEST reqMktData {'reqId': 1005, 'contract': 139881771409552: 0,TSLA,STK,,,0,,,SMART,,USD,,,False,,,,combo:, 'genericTickList': '', 'snapshot': False, 'regulatorySnapshot': False, 'mktDataOptions': []}
2024-12-13 02:01:32,368 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,368 - INFO - SENDING reqMktData b'\x00\x00\x0011\x0011\x001005\x000\x00TSLA\x00STK\x00\x000.0\x00\x00\x00SMART\x00\x00USD\x00\x00\x000\x00\x000\x000\x00\x00'
2024-12-13 02:01:32,368 - DEBUG - acquiring lock
2024-12-13 02:01:32,368 - DEBUG - acquired lock
2024-12-13 02:01:32,368 - DEBUG - releasing lock
2024-12-13 02:01:32,368 - DEBUG - release lock
2024-12-13 02:01:32,368 - DEBUG - sendMsg: sent: 53
2024-12-13 02:01:32,368 - DEBUG - Requested market data for TSLA with ReqId: 1005
2024-12-13 02:01:32,368 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,368 - DEBUG - conn:1 queue.sz:3
2024-12-13 02:01:32,368 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,368 - DEBUG - fields (b'4', b'2', b'-1', b'2104', b'Market data farm connection is OK:usfarm', b'')
2024-12-13 02:01:32,368 - DEBUG - decode <class 'int'> b'2'
2024-12-13 02:01:32,368 - DEBUG - decode <class 'int'> b'-1'
2024-12-13 02:01:32,368 - DEBUG - decode <class 'int'> b'2104'
2024-12-13 02:01:32,368 - DEBUG - decode <class 'str'> b'Market data farm connection is OK:usfarm'
2024-12-13 02:01:32,369 - DEBUG - decode <class 'str'> b''
2024-12-13 02:01:32,369 - ERROR - Error. ReqId: -1, Code: 2104, Msg: Market data farm connection is OK:usfarm, Advanced Order Reject JSON:
2024-12-13 02:01:32,369 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,369 - DEBUG - conn:1 queue.sz:2
2024-12-13 02:01:32,369 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,369 - DEBUG - fields (b'4', b'2', b'-1', b'2107', b'HMDS data farm connection is inactive but should be available upon demand.ushmds', b'')
2024-12-13 02:01:32,369 - DEBUG - decode <class 'int'> b'2'
2024-12-13 02:01:32,369 - DEBUG - decode <class 'int'> b'-1'
2024-12-13 02:01:32,369 - DEBUG - decode <class 'int'> b'2107'
2024-12-13 02:01:32,369 - DEBUG - decode <class 'str'> b'HMDS data farm connection is inactive but should be available upon demand.ushmds'
2024-12-13 02:01:32,369 - DEBUG - decode <class 'str'> b''
2024-12-13 02:01:32,369 - ERROR - Error. ReqId: -1, Code: 2107, Msg: HMDS data farm connection is inactive but should be available upon demand.ushmds, Advanced Order Reject JSON:
2024-12-13 02:01:32,369 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,369 - DEBUG - conn:1 queue.sz:1
2024-12-13 02:01:32,369 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,369 - DEBUG - fields (b'4', b'2', b'-1', b'2158', b'Sec-def data farm connection is OK:secdefnj', b'')
2024-12-13 02:01:32,369 - DEBUG - decode <class 'int'> b'2'
2024-12-13 02:01:32,369 - DEBUG - decode <class 'int'> b'-1'
2024-12-13 02:01:32,369 - DEBUG - decode <class 'int'> b'2158'
2024-12-13 02:01:32,369 - DEBUG - decode <class 'str'> b'Sec-def data farm connection is OK:secdefnj'
2024-12-13 02:01:32,369 - DEBUG - decode <class 'str'> b''
2024-12-13 02:01:32,369 - ERROR - Error. ReqId: -1, Code: 2158, Msg: Sec-def data farm connection is OK:secdefnj, Advanced Order Reject JSON:
2024-12-13 02:01:32,369 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,369 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:32,369 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,433 - DEBUG - len 203 raw:b"\x00\x00\x00\xc74\x002\x001001\x0010089\x00Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.AAPL NASDAQ.NMS/TOP/ALL\x00\x00"|
2024-12-13 02:01:32,433 - DEBUG - reader loop, recvd size 203
2024-12-13 02:01:32,433 - DEBUG - read_msg: size: 199
2024-12-13 02:01:32,433 - DEBUG - size:199 msg.size:199 msg:|b''| buf:||
2024-12-13 02:01:32,434 - DEBUG - fields (b'4', b'2', b'1001', b'10089', b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.AAPL NASDAQ.NMS/TOP/ALL", b'')
2024-12-13 02:01:32,434 - DEBUG - decode <class 'int'> b'2'
2024-12-13 02:01:32,434 - DEBUG - decode <class 'int'> b'1001'
2024-12-13 02:01:32,434 - DEBUG - decode <class 'int'> b'10089'
2024-12-13 02:01:32,434 - DEBUG - decode <class 'str'> b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.AAPL NASDAQ.NMS/TOP/ALL"
2024-12-13 02:01:32,434 - DEBUG - decode <class 'str'> b''
2024-12-13 02:01:32,434 - ERROR - Error. ReqId: 1001, Code: 10089, Msg: Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.AAPL NASDAQ.NMS/TOP/ALL, Advanced Order Reject JSON:
2024-12-13 02:01:32,434 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,434 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:32,434 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,634 - DEBUG - queue.get: empty
2024-12-13 02:01:32,634 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,634 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:32,634 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,685 - DEBUG - len 203 raw:b"\x00\x00\x00\xc74\x002\x001002\x0010089\x00Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.MSFT NASDAQ.NMS/TOP/ALL\x00\x00"|
2024-12-13 02:01:32,685 - DEBUG - reader loop, recvd size 203
2024-12-13 02:01:32,685 - DEBUG - read_msg: size: 199
2024-12-13 02:01:32,685 - DEBUG - size:199 msg.size:199 msg:|b''| buf:||
2024-12-13 02:01:32,685 - DEBUG - fields (b'4', b'2', b'1002', b'10089', b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.MSFT NASDAQ.NMS/TOP/ALL", b'')
2024-12-13 02:01:32,685 - DEBUG - decode <class 'int'> b'2'
2024-12-13 02:01:32,685 - DEBUG - decode <class 'int'> b'1002'
2024-12-13 02:01:32,685 - DEBUG - decode <class 'int'> b'10089'
2024-12-13 02:01:32,685 - DEBUG - decode <class 'str'> b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.MSFT NASDAQ.NMS/TOP/ALL"
2024-12-13 02:01:32,685 - DEBUG - decode <class 'str'> b''
2024-12-13 02:01:32,685 - ERROR - Error. ReqId: 1002, Code: 10089, Msg: Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.MSFT NASDAQ.NMS/TOP/ALL, Advanced Order Reject JSON:
2024-12-13 02:01:32,685 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,685 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:32,685 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,710 - DEBUG - len 204 raw:b"\x00\x00\x00\xc84\x002\x001003\x0010089\x00Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.GOOGL NASDAQ.NMS/TOP/ALL\x00\x00"|
2024-12-13 02:01:32,710 - DEBUG - reader loop, recvd size 204
2024-12-13 02:01:32,710 - DEBUG - read_msg: size: 200
2024-12-13 02:01:32,710 - DEBUG - size:200 msg.size:200 msg:|b''| buf:||
2024-12-13 02:01:32,710 - DEBUG - fields (b'4', b'2', b'1003', b'10089', b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.GOOGL NASDAQ.NMS/TOP/ALL", b'')
2024-12-13 02:01:32,710 - DEBUG - decode <class 'int'> b'2'
2024-12-13 02:01:32,710 - DEBUG - decode <class 'int'> b'1003'
2024-12-13 02:01:32,710 - DEBUG - decode <class 'int'> b'10089'
2024-12-13 02:01:32,710 - DEBUG - decode <class 'str'> b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.GOOGL NASDAQ.NMS/TOP/ALL"
2024-12-13 02:01:32,710 - DEBUG - decode <class 'str'> b''
2024-12-13 02:01:32,710 - ERROR - Error. ReqId: 1003, Code: 10089, Msg: Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.GOOGL NASDAQ.NMS/TOP/ALL, Advanced Order Reject JSON:
2024-12-13 02:01:32,710 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,710 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:32,710 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,711 - DEBUG - len 203 raw:b"\x00\x00\x00\xc74\x002\x001004\x0010089\x00Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.AMZN NASDAQ.NMS/TOP/ALL\x00\x00"|
2024-12-13 02:01:32,711 - DEBUG - reader loop, recvd size 203
2024-12-13 02:01:32,711 - DEBUG - read_msg: size: 199
2024-12-13 02:01:32,711 - DEBUG - size:199 msg.size:199 msg:|b''| buf:||
2024-12-13 02:01:32,711 - DEBUG - fields (b'4', b'2', b'1004', b'10089', b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.AMZN NASDAQ.NMS/TOP/ALL", b'')
2024-12-13 02:01:32,712 - DEBUG - decode <class 'int'> b'2'
2024-12-13 02:01:32,712 - DEBUG - decode <class 'int'> b'1004'
2024-12-13 02:01:32,712 - DEBUG - decode <class 'int'> b'10089'
2024-12-13 02:01:32,712 - DEBUG - decode <class 'str'> b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.AMZN NASDAQ.NMS/TOP/ALL"
2024-12-13 02:01:32,712 - DEBUG - decode <class 'str'> b''
2024-12-13 02:01:32,712 - ERROR - Error. ReqId: 1004, Code: 10089, Msg: Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.AMZN NASDAQ.NMS/TOP/ALL, Advanced Order Reject JSON:
2024-12-13 02:01:32,712 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,712 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:32,712 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,912 - DEBUG - queue.get: empty
2024-12-13 02:01:32,912 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,912 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:32,912 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,934 - DEBUG - len 203 raw:b"\x00\x00\x00\xc74\x002\x001005\x0010089\x00Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.TSLA NASDAQ.NMS/TOP/ALL\x00\x00"|
2024-12-13 02:01:32,934 - DEBUG - reader loop, recvd size 203
2024-12-13 02:01:32,934 - DEBUG - read_msg: size: 199
2024-12-13 02:01:32,934 - DEBUG - size:199 msg.size:199 msg:|b''| buf:||
2024-12-13 02:01:32,934 - DEBUG - fields (b'4', b'2', b'1005', b'10089', b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.TSLA NASDAQ.NMS/TOP/ALL", b'')
2024-12-13 02:01:32,934 - DEBUG - decode <class 'int'> b'2'
2024-12-13 02:01:32,934 - DEBUG - decode <class 'int'> b'1005'
2024-12-13 02:01:32,934 - DEBUG - decode <class 'int'> b'10089'
2024-12-13 02:01:32,934 - DEBUG - decode <class 'str'> b"Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.TSLA NASDAQ.NMS/TOP/ALL"
2024-12-13 02:01:32,934 - DEBUG - decode <class 'str'> b''
2024-12-13 02:01:32,934 - ERROR - Error. ReqId: 1005, Code: 10089, Msg: Requested market data requires additional subscription for API. See link in 'Market Data Connections' dialog for more details.Delayed market data is available.TSLA NASDAQ.NMS/TOP/ALL, Advanced Order Reject JSON:
2024-12-13 02:01:32,935 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:32,935 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:32,935 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,135 - DEBUG - queue.get: empty
2024-12-13 02:01:33,135 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,135 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:33,135 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,335 - DEBUG - queue.get: empty
2024-12-13 02:01:33,335 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,335 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:33,335 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,536 - DEBUG - queue.get: empty
2024-12-13 02:01:33,536 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,536 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:33,536 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,736 - DEBUG - queue.get: empty
2024-12-13 02:01:33,736 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,736 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:33,736 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,935 - DEBUG - socket timeout from recvMsg (<class 'TimeoutError'>, TimeoutError('timed out'), <traceback object at 0x7f38c34ba040>)
2024-12-13 02:01:33,935 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:33,936 - DEBUG - queue.get: empty
2024-12-13 02:01:33,936 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:33,936 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:33,936 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,137 - DEBUG - queue.get: empty
2024-12-13 02:01:34,137 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,137 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:34,137 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,337 - DEBUG - queue.get: empty
2024-12-13 02:01:34,337 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,337 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:34,337 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,537 - DEBUG - queue.get: empty
2024-12-13 02:01:34,538 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,538 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:34,538 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,738 - DEBUG - queue.get: empty
2024-12-13 02:01:34,738 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,738 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:34,738 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,937 - DEBUG - socket timeout from recvMsg (<class 'TimeoutError'>, TimeoutError('timed out'), <traceback object at 0x7f38c34b9ec0>)
2024-12-13 02:01:34,937 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:34,938 - DEBUG - queue.get: empty
2024-12-13 02:01:34,938 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:34,938 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:34,938 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,139 - DEBUG - queue.get: empty
2024-12-13 02:01:35,139 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,139 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:35,139 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,339 - DEBUG - queue.get: empty
2024-12-13 02:01:35,339 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,339 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:35,339 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,539 - DEBUG - queue.get: empty
2024-12-13 02:01:35,539 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,539 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:35,539 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,740 - DEBUG - queue.get: empty
2024-12-13 02:01:35,740 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,740 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:35,740 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,938 - DEBUG - socket timeout from recvMsg (<class 'TimeoutError'>, TimeoutError('timed out'), <traceback object at 0x7f38c34ba1c0>)
2024-12-13 02:01:35,938 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:35,940 - DEBUG - queue.get: empty
2024-12-13 02:01:35,940 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:35,940 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:35,940 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,141 - DEBUG - queue.get: empty
2024-12-13 02:01:36,141 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,141 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:36,141 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,341 - DEBUG - queue.get: empty
2024-12-13 02:01:36,341 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,341 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:36,341 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,541 - DEBUG - queue.get: empty
2024-12-13 02:01:36,541 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,541 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:36,541 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,742 - DEBUG - queue.get: empty
2024-12-13 02:01:36,742 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,742 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:36,742 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,939 - DEBUG - socket timeout from recvMsg (<class 'TimeoutError'>, TimeoutError('timed out'), <traceback object at 0x7f38c34ba040>)
2024-12-13 02:01:36,939 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:36,942 - DEBUG - queue.get: empty
2024-12-13 02:01:36,942 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:36,942 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:36,942 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,143 - DEBUG - queue.get: empty
2024-12-13 02:01:37,143 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,143 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:37,143 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,343 - DEBUG - queue.get: empty
2024-12-13 02:01:37,343 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,343 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:37,343 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,543 - DEBUG - queue.get: empty
2024-12-13 02:01:37,543 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,543 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:37,543 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,744 - DEBUG - queue.get: empty
2024-12-13 02:01:37,744 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,744 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:37,744 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,941 - DEBUG - socket timeout from recvMsg (<class 'TimeoutError'>, TimeoutError('timed out'), <traceback object at 0x7f38c34b9ec0>)
2024-12-13 02:01:37,941 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:37,944 - DEBUG - queue.get: empty
2024-12-13 02:01:37,944 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:37,944 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:37,944 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,144 - DEBUG - queue.get: empty
2024-12-13 02:01:38,144 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,144 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:38,145 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,345 - DEBUG - queue.get: empty
2024-12-13 02:01:38,345 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,345 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:38,345 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,545 - DEBUG - queue.get: empty
2024-12-13 02:01:38,545 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,545 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:38,545 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,746 - DEBUG - queue.get: empty
2024-12-13 02:01:38,746 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,746 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:38,746 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,942 - DEBUG - socket timeout from recvMsg (<class 'TimeoutError'>, TimeoutError('timed out'), <traceback object at 0x7f38c34ba1c0>)
2024-12-13 02:01:38,942 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:38,946 - DEBUG - queue.get: empty
2024-12-13 02:01:38,946 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:38,946 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:38,946 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,146 - DEBUG - queue.get: empty
2024-12-13 02:01:39,146 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,146 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:39,146 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,346 - DEBUG - queue.get: empty
2024-12-13 02:01:39,347 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,347 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:39,347 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,547 - DEBUG - queue.get: empty
2024-12-13 02:01:39,547 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,547 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:39,547 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,747 - DEBUG - queue.get: empty
2024-12-13 02:01:39,747 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,747 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:39,747 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,942 - DEBUG - socket timeout from recvMsg (<class 'TimeoutError'>, TimeoutError('timed out'), <traceback object at 0x7f38c34ba040>)
2024-12-13 02:01:39,942 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:39,948 - DEBUG - queue.get: empty
2024-12-13 02:01:39,948 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:39,948 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:39,948 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,148 - DEBUG - queue.get: empty
2024-12-13 02:01:40,148 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,148 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:40,148 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,348 - DEBUG - queue.get: empty
2024-12-13 02:01:40,348 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,348 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:40,348 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,549 - DEBUG - queue.get: empty
2024-12-13 02:01:40,549 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,549 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:40,549 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,749 - DEBUG - queue.get: empty
2024-12-13 02:01:40,749 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,749 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:40,749 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,943 - DEBUG - socket timeout from recvMsg (<class 'TimeoutError'>, TimeoutError('timed out'), <traceback object at 0x7f38c34b9ec0>)
2024-12-13 02:01:40,943 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:40,949 - DEBUG - queue.get: empty
2024-12-13 02:01:40,949 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:40,949 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:40,949 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,150 - DEBUG - queue.get: empty
2024-12-13 02:01:41,150 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,150 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:41,150 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,350 - DEBUG - queue.get: empty
2024-12-13 02:01:41,350 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,350 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:41,350 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,550 - DEBUG - queue.get: empty
2024-12-13 02:01:41,550 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,550 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:41,550 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,751 - DEBUG - queue.get: empty
2024-12-13 02:01:41,751 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,751 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:41,751 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,945 - DEBUG - socket timeout from recvMsg (<class 'TimeoutError'>, TimeoutError('timed out'), <traceback object at 0x7f38c34ba1c0>)
2024-12-13 02:01:41,945 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:41,951 - DEBUG - queue.get: empty
2024-12-13 02:01:41,951 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:41,951 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:41,951 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:42,152 - DEBUG - queue.get: empty
2024-12-13 02:01:42,152 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:42,152 - DEBUG - conn:1 queue.sz:0
2024-12-13 02:01:42,152 - DEBUG - 139881771299920 isConn: 2, connConnected: True
2024-12-13 02:01:42,322 - WARNING - Timeout while waiting for market data.
2024-12-13 02:01:42,322 - DEBUG - 139881771299920 connState: 2 -> 0
2024-12-13 02:01:42,322 - INFO - disconnecting
2024-12-13 02:01:42,322 - DEBUG - disconnecting
2024-12-13 02:01:42,322 - DEBUG - disconnected
2024-12-13 02:01:42,322 - INFO - Connection to IB Gateway closed.
2024-12-13 02:01:42,323 - DEBUG - 139881771299920 connState: None -> 0
2024-12-13 02:01:42,352 - DEBUG - queue.get: empty
2024-12-13 02:01:42,352 - DEBUG - 139881771299920 isConn: 0, connConnected: None
2024-12-13 02:01:42,352 - DEBUG - conn:0 queue.sz:0
2024-12-13 02:01:42,352 - DEBUG - 139881771299920 isConn: 0, connConnected: None
2024-12-13 02:01:42,352 - DEBUG - 139881771299920 connState: 0 -> 0
2024-12-13 02:01:42,946 - DEBUG - socket broken, disconnecting
2024-12-13 02:01:42,946 - DEBUG - reader loop, recvd size 0
2024-12-13 02:01:42,946 - DEBUG - EReader thread finished
2024-12-13 02:01:44,323 - INFO - TestStockRetriever completed.

238
src/MidasV1/tests/test_stock_retriever.py Normal file
View File

@@ -0,0 +1,238 @@
# test_stock_retriever.py
"""
========================================================================
# README
#
# Program: test_stock_retriever.py
#
# Description:
# This script serves as a testing tool for the MidasV1 Trading Bot.
# It connects to the Interactive Brokers (IB) Gateway, retrieves real-time
# market data for a predefined list of stock symbols, and applies internal
# criteria to filter the stocks based on share price and trading volume.
#
# Features:
# - Connects to IB Gateway on a specified host and port.
# - Requests market data (last price and volume) for a list of stock symbols.
# - Applies filtering criteria to identify stocks that meet minimum share price
# and trading volume requirements.
# - Provides colored console outputs for better readability and user feedback.
# - Handles graceful shutdown on interrupt signals and error conditions.
# - Implements robust error handling to manage API connection issues and data retrieval problems.
#
# Usage:
# Run the script from the command line:
# python test_stock_retriever.py
#
# The script will attempt to connect to IB Gateway, request market data, and display
# the list of stocks that meet the specified criteria. If no stocks meet the criteria,
# it will notify the user accordingly.
#
# Enhancements:
# - Added colored outputs using ANSI escape codes (green for pass, red for fail).
# - Implemented signal handling for graceful exits on interrupts.
# - Enhanced error handling for various API and connection errors.
# - Improved logging for detailed traceability.
#
# Coded by: kleinpanic 2024
========================================================================
"""
import logging
import threading
import time
import signal
import sys
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
from ibapi.ticktype import TickTypeEnum
# ANSI color codes for enhanced console outputs
GREEN = "\033[92m"
RED = "\033[91m"
YELLOW = "\033[93m"
BLUE = "\033[94m"
MAGENTA = "\033[95m"
RESET = "\033[0m"
# Configure logging
logging.basicConfig(
filename='test_stock_retriever.log',
filemode='w',
level=logging.DEBUG,
format='%(asctime)s - %(levelname)s - %(message)s'
)
class TestStockRetriever(EWrapper, EClient):
def __init__(self, symbols, criteria):
EClient.__init__(self, self)
self.symbols = symbols
self.criteria = criteria
self.stock_data = {}
self.data_event = threading.Event()
self.logger = logging.getLogger('MidasV1.TestStockRetriever')
def error(self, reqId, errorCode, errorString, advancedOrderRejectJson=""):
logging.error(f"Error. ReqId: {reqId}, Code: {errorCode}, Msg: {errorString}, Advanced Order Reject JSON: {advancedOrderRejectJson}")
print(f"{RED}Error. ReqId: {reqId}, Code: {errorCode}, Msg: {errorString}, Advanced Order Reject JSON: {advancedOrderRejectJson}{RESET}")
def tickPrice(self, reqId, tickType, price, attrib):
# We are interested in LAST price
if tickType == TickTypeEnum.LAST:
symbol = self.symbols.get(reqId, None)
if symbol:
self.stock_data.setdefault(symbol, {})['last_price'] = price
logging.debug(f"TickPrice. ReqId: {reqId}, Symbol: {symbol}, Last Price: {price}")
# Check if both last_price and volume are received
if 'volume' in self.stock_data[symbol]:
self.data_event.set()
print(f"{GREEN}Received last price for {symbol}: ${price}{RESET}")
def tickSize(self, reqId, tickType, size):
# We are interested in VOLUME
if tickType == TickTypeEnum.VOLUME:
symbol = self.symbols.get(reqId, None)
if symbol:
self.stock_data.setdefault(symbol, {})['volume'] = size
logging.debug(f"TickSize. ReqId: {reqId}, Symbol: {symbol}, Volume: {size}")
# Check if both last_price and volume are received
if 'last_price' in self.stock_data[symbol]:
self.data_event.set()
print(f"{GREEN}Received volume for {symbol}: {size}{RESET}")
def tickString(self, reqId, tickType, value):
# Optionally handle other tick types
pass
def tickGeneric(self, reqId, tickType, value):
# Optionally handle other tick types
pass
def nextValidId(self, orderId):
# Start requesting market data once the next valid order ID is received
logging.info(f"NextValidId received: {orderId}")
self.request_market_data()
def request_market_data(self):
logging.info("Requesting market data for symbols...")
print(f"{BLUE}Requesting market data for symbols...{RESET}")
for reqId, symbol in self.symbols.items():
contract = self.create_contract(symbol)
self.reqMktData(reqId, contract, "", False, False, [])
logging.debug(f"Requested market data for {symbol} with ReqId: {reqId}")
print(f"{BLUE}Requested market data for {symbol} with ReqId: {reqId}{RESET}")
def create_contract(self, symbol):
contract = Contract()
contract.symbol = symbol
contract.secType = "STK"
contract.exchange = "SMART"
contract.currency = "USD"
return contract
def marketDataType(self, reqId, marketDataType):
# Optionally handle different market data types
logging.debug(f"MarketDataType. ReqId: {reqId}, Type: {marketDataType}")
def connectionClosed(self):
logging.info("Connection to IB Gateway closed.")
print(f"{YELLOW}Connection to IB Gateway closed.{RESET}")
def run_retriever(self):
# Start the socket in a thread
api_thread = threading.Thread(target=self.run, daemon=True)
api_thread.start()
# Wait for data to be collected or timeout after 10 seconds
if self.data_event.wait(timeout=10):
logging.info("Market data retrieved successfully.")
print(f"{GREEN}Market data retrieved successfully.{RESET}")
else:
logging.warning("Timeout while waiting for market data.")
print(f"{RED}Timeout while waiting for market data.{RESET}")
# Disconnect after data retrieval
self.disconnect()
def stop(self):
self.disconnect()
def signal_handler(sig, frame, app):
"""
Handles incoming signals for graceful shutdown.
Args:
sig (int): Signal number.
frame: Current stack frame.
app (TestStockRetriever): The running application instance.
"""
logger = logging.getLogger('MidasV1.TestStockRetriever')
logger.error("Interrupt received. Shutting down gracefully...")
print(f"{RED}Interrupt received. Shutting down gracefully...{RESET}")
app.disconnect()
sys.exit(0)
def main():
# Define a list of stock symbols to retrieve data for
symbols_to_test = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'TSLA']
# Assign a unique ReqId to each symbol
symbols = {1001 + idx: symbol for idx, symbol in enumerate(symbols_to_test)}
# Define internal criteria for filtering stocks
criteria = {
'min_share_price': 50.0, # Minimum share price in USD
'min_volume': 1000000 # Minimum trading volume
}
# Instantiate the TestStockRetriever
app = TestStockRetriever(symbols, criteria)
# Register the signal handler for graceful shutdown
signal.signal(signal.SIGINT, lambda sig, frame: signal_handler(sig, frame, app))
signal.signal(signal.SIGTERM, lambda sig, frame: signal_handler(sig, frame, app))
# Connect to IB Gateway or TWS
# Update the port and clientId based on your setup
app.connect("127.0.0.1", 4002, clientId=1) # Use 4002 for IB Gateway Simulated Trading
# Start the data retrieval process
app.run_retriever()
# Allow some time for data to be processed
time.sleep(2)
# Apply criteria to filter stocks
filtered_stocks = []
for symbol, data in app.stock_data.items():
last_price = data.get('last_price', 0)
volume = data.get('volume', 0)
if last_price >= criteria['min_share_price'] and volume >= criteria['min_volume']:
filtered_stocks.append({
'symbol': symbol,
'last_price': last_price,
'volume': volume
})
logging.info(f"Stock {symbol} meets criteria: Price=${last_price}, Volume={volume}")
else:
logging.info(f"Stock {symbol} does not meet criteria: Price=${last_price}, Volume={volume}")
# Display the filtered stocks
if filtered_stocks:
print(f"{GREEN}\nStocks meeting the criteria:{RESET}")
print("-----------------------------")
for stock in filtered_stocks:
print(f"Symbol: {stock['symbol']}, Last Price: ${stock['last_price']:.2f}, Volume: {stock['volume']}")
print("-----------------------------\n")
else:
print(f"{RED}\nNo stocks met the specified criteria.\n{RESET}")
# Log the end of the test
logging.info("TestStockRetriever completed.")
if __name__ == "__main__":
main()

0
src/API/API_1.ipynb → src/griffin-stuff/API/API_1.ipynb
View File

0
src/API/API_2 (1).ipynb → src/griffin-stuff/API/API_2 (1).ipynb
View File

0
src/API/Trading_Bot_Development_Strategy (1).docx → src/griffin-stuff/API/Trading_Bot_Development_Strategy (1).docx
View File

0
src/GUSHTradingBotV1.0.py → src/griffin-stuff/GUSHTradingBotV1.0.py
View File

0
src/IBKR/3_month_testing_data.csv → src/griffin-stuff/IBKR/3_month_testing_data.csv
View File

Can't render this file because it is too large.

0
src/IBKR/3_years_training_data.csv → src/griffin-stuff/IBKR/3_years_training_data.csv
View File

Can't render this file because it is too large.

0
src/IBKR/predict_price.py → src/griffin-stuff/IBKR/predict_price.py
View File

0
src/IBKR/requirements.txt → src/griffin-stuff/IBKR/requirements.txt
View File