chore: Refactor data handling logic and implement ppxt call and respo…

…nse handling

openaicall.py

@@ -37,27 +37,49 @@ def create_prompt(income_statement, balance_sheet):
     return prompt
 
 def get_predictions(company_code):
+    # Create a dataframe to store the predictions
+    predictions = pd.DataFrame()
+
     # Fetch the data
     income_statement = retrieve_income_with_lenght(company_code)
     balance_sheet = retrieve_balance_with_lenght(company_code)
 
     bs_len = balance_sheet['len']
-    bs_numbers=balance_sheet['balance_sheet']
+    bs_numbers = balance_sheet['balance_sheet']
 
     is_len = income_statement['len']
-    is_numbers=income_statement['income_statement']
-
-    if bs_len == is_len:
-        print("Data is consistent")
-    else:
+    is_numbers = income_statement['income_statement']
+    #print(is_numbers)
+    print("income_statement len: " + str(is_len))
+    print()
+    print("income_statement years: " + str(is_numbers))
+    print()
+    print("income_statement years index: " + str(is_numbers.index))
+
+
+    print("balance_sheet years: " + str(bs_len))
+    print()
+    print("balance_sheet years: " + str(bs_numbers))
+    print()
+    print("balance_sheet years index: " + str(bs_numbers.index))
+
+    is_numbers.index = is_numbers.index.astype(int)
+    bs_numbers.index = bs_numbers.index.astype(int)
+
+    if bs_len != is_len:
         print("Data is inconsistent")
+        return {"error": "Data inconsistency between balance sheet and income statement lengths"}
+
     # Determine available years for prediction
-    years = bs_numbers.index.astype(int)
-    predictions = []
+    years = bs_numbers.index
+    predictions_list = []
 
     for i in range(5, bs_len):
         historical_years = years[i-5:i]
-        prediction_year = years[i]
+        if not historical_years.isin(is_numbers.index).all():
+            print(f"Missing data for years: {historical_years[~historical_years.isin(is_numbers.index)]}")
+            continue
+
 
         # Create the prompt
         historical_income = is_numbers.loc[historical_years]
@@ -66,10 +88,9 @@ def get_predictions(company_code):
 
         # Use OpenAI API to analyze the financials
         response = openai.chat.completions.create(
-            model="gpt-4o",
+            model="gpt-4-turbo",
             temperature=0,
             top_p=1,
-            logprobs=True,
             messages=[
                 {"role": "system", "content": "You are a financial analyst."},
                 {"role": "user", "content": prompt}
@@ -78,8 +99,22 @@ def get_predictions(company_code):
         )
 
         resp = response.choices[0].message.content.strip()
-
-    return resp
+        cleaned_json_string = resp.strip('```json\n')
+
+        # Convert the cleaned string to a JSON object
+        try:
+            json_obj = json.loads(cleaned_json_string)
+            predictions_list.append(json_obj)
+            #print year processed
+            print(f"Processed year: {json_obj['Year']}")
+        except json.JSONDecodeError as e:
+            print(f"Failed to decode JSON: {e}")
+
+    # Convert the list of JSON objects to a DataFrame
+    if predictions_list:
+        predictions = pd.DataFrame(predictions_list)
+
+    return predictions
 
 def get_predictions_ppxt(company_code):
     # Fetch the data

tests/outputx.txt

@@ -8,22 +8,27 @@ Enter output type (A/B): A
 x^X2024-06-21 19:19:24,467 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
 ^R
 
-çç^X^Xxx2024-06-21 19:19:41,316 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
-2024-06-21 19:19:54,730 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
+ddmw-py3.12pedro@MacBook-Pro % source ~/.zshrc
 2024-06-21 19:20:12,242 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
 2024-06-21 19:20:27,704 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
-2024-06-21 19:20:44,730 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
-Prediction for 2019: next_year; increase
 
 Panel A. Trend Analysis:
-The trend analysis shows consistent increases in several sectors which indicate a promising future for the company. There is a consistent rise in 'net_income' from 2014 to 2018 with a noticeable spike in 2018. Moreover, 'gross_income' and 'ebit' also show a consistent upward trend. 'Net_sales' increase steadily, indicating the company's capacity to generate revenue is improving.
+The trend analysis shows consistent increases in several sectors which indicate a promising future for the company. There is a consistent rise in 'net_income' from 2014 
+to 2018 with a noticeable spike in 2018. Moreover, 'gross_income' and 'ebit' also 
+show a consistent upward trend. 'Net_sales' increase steadily, indicating the company's capacity to generate revenue is improving.
 
 Panel B. Ratio Analysis:
-The debt-to-equity ratio seems to be decreasing over the years since the 'equity' number is rising at a faster pace than 'loans'. The company's current ratio (current_assets/current_liabilities) is improving, meaning the company is becoming more capable of paying off its debts. Another indicator of financial health, the net profit margin, (net_income/net_sales) has also been showing an increase.
+The debt-to-equity ratio seems to be decreasing over the years since the 'equity' number is rising at a faster pace than 'loans'. 
+The company's current ratio (current_assets/current_liabilities) is improving, meaning the company is becoming more capable of paying off its debts. 
+Another indicator of financial health, the net profit margin, (net_income/net_sales) has also been showing an increase.
 
 Panel C. Rationale:
-The reason for predicting an increase in the company's earnings next year lies in the upward trend in the company's income and sales and its improving financial ratios. It indicates that the company is not only successful in escalating its revenues but is also managing its costs and liabilities efficiently. Its solid performance in the past suggests it has the potential to continue this growth trajectory. However, external factors like market conditions and business climate could significantly impact this forecast.
-Prediction for 2020: next_year; increase
+The reason for predicting an increase in the company's earnings next year lies in the upward trend in the company's income and sales and its improving financial ratios. 
+It indicates that the company is not only successful in escalating its revenues but is also managing its costs and liabilities efficiently. 
+Its solid performance in the past suggests it has the potential to continue this growth trajectory. 
+However, external factors like market conditions and business climate could significantly impact this forecast.
+Prediction for 2020: next_year; increase; magnitude: moderate;confidence score	: 0.69
+---- Rolling logprob avg:0.73; Rolling F-Score: 0.488
 
 Panel A. Trend Analysis:
 Upon observing the income statement and the balance sheet, it's clear that the company has shown a consistent upward trend in net sales, gross income, and net income over the five-year period (2015-2019). Net sales have increased from approximately 1 billion to 3.8 billion, along with the gross income and net income. This suggests a positive trend in the company's financial performance. Additionally, on the balance sheet, total assets and equity have also increased significantly, indicating growth and consistent reinvestment in the company's equity.