SP 500 List python script crashes

2024/10/13 3:29:29

So I have been following a youtube tutorial on Python finance and since Yahoo has now closed its doors to the financial market, it has caused a few dwelling problems.

I run this code

    import bs4 as bs
import datetime as dt 
import os
import pandas as pd
import pandas_datareader.data as web
import pickle
import requests
from pandas_datareader import data as pdr
import fix_yahoo_finance as yfdef save_sp500_tickers():resp = requests.get('https://en.wikipedia.org       /wiki/List_of_S%26P_500_companies')soup = bs.BeautifulSoup(resp.text, "lxml")table = soup.find('table', {'class':'wikitable sortable'})tickers = []for row in table.findAll('tr')[1:]:ticker = row.findAll('td')[0].texttickers.append(ticker)with open("sp500tickers.pickle", "wb") as f:pickle.dump(ticker, f)print(tickers)return tickers# save_sp500_tickers()def get_data_from_yahoo(reload_sp500=False):if reload_sp500:tickers = save_sp500_tickers()else:with open("sp500tickers.pickle", "rb") as f:tickers = pickle.load(f)if not os.path.exists('stock_dfs'):os.makedirs('stock_dfs')start = dt.datetime(2000, 1, 1)end = dt.datetime(2017, 8, 24)for ticker in tickers:if not os.path.exists('stock_dfs/{}.csv'.format(ticker)):data = pdr.get_data_yahoo(ticker, start, end)df.to_csv('stock_dfs/{}.csv'.format(ticker))else:print('Already have {}'.format(ticker))get_data_from_yahoo()

and it crashes with a few erros, not just one. The first error is that I should override pandas data reader.

DeprecationWarning: 
Auto-overriding of pandas_datareader's get_data_yahoo() is deprecated and will be removed in future versions.
Use pdr_override() to explicitly override it.
DeprecationWarning)  

How do I override it? I really don't know how to, I am new to Python - sorry for being a noob.

Then we have this:

    get_data_from_yahoo()File "C:\Users\Mehdi\Desktop\Python finance\SP500_List.py", line 36, in get_data_from_yahootickers = pickle.load(f)

I really dont understand why this happens, because I have checked my code with the Youtuber and they match. So some pointer would be appriciated.

Finally, I have this error:

  EOFError: Ran out of input

which I also don't know what it means.

To add to that, I have installed the 'fix_yahoo_finance' package and tried that with the new code as well and it still does not work.

Any help is appriciated. Thanks :)

Full error list:

C:\Users\Mehdi\AppData\Local\Programs\Python\Python36-32\lib\site-packages\fix_yahoo_finance\__init__.py:43: DeprecationWarning: Auto-overriding of pandas_datareader's get_data_yahoo() is deprecated and will be removed in future versions.Use pdr_override() to explicitly override it.DeprecationWarning)
Traceback (most recent call last):File "C:\Users\Mehdi\Desktop\Python finance\SP500_List.py", line 51, in <module>get_data_from_yahoo()File "C:\Users\Mehdi\Desktop\Python finance\SP500_List.py", line 36, in get_data_from_yahootickers = pickle.load(f)
EOFError: Ran out of input
[Finished in 3.1s with exit code 1]
[shell_cmd: python -u "C:\Users\Mehdi\Desktop\Python finance\SP500_List.py"]
[dir: C:\Users\Mehdi\Desktop\Python finance]
[path: C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\Users\Mehdi\AppData\Local\Programs\Python\Python36-32\Scripts\;C:\Users\Mehdi\AppData\Local\Programs\Python\Python36-32\;C:\Users\Mehdi\AppData\Local\Microsoft\WindowsApps;C:\Python36\Scripts;C:\Users\Mehdi\AppData\Roaming\Dashlane\4.8.5.35155\bin\Firefox_Extension\{442718d9-475e-452a-b3e1-fb1ee16b8e9f}\components;C:\Users\Mehdi\AppData\Roaming\Dashlane\4.8.5.35155\ucrt]
Answer

You have two mistakes in your code:

  1. In line 22 in function save_sp500_tickers(), instead of this:

    with open("sp500tickers.pickle", "wb") as f:pickle.dump(ticker, f)
    

    it should be:

    with open("sp500tickers.pickle", "wb") as f:pickle.dump(tickers, f)
    

    So it is tickers instead of ticker

  2. In line 47 in function get_data_from_yahoo(), instead of this:

    if not os.path.exists('stock_dfs/{}.csv'.format(ticker)):data = pdr.get_data_yahoo(ticker, start, end)df.to_csv('stock_dfs/{}.csv'.format(ticker))
    

    It should be:

    if not os.path.exists('stock_dfs/{}.csv'.format(ticker)):data = pdr.get_data_yahoo(ticker, start, end)data.to_csv('stock_dfs/{}.csv'.format(ticker))
    

    You need to use data instead of df (in video was used df = web.DataReader(ticker, 'yahoo', start, end) which you change in data = pdr.get_data_yahoo(ticker, start, end) but you forgot to change df.to_csv('stock_dfs/{}.csv'.format(ticker)) to data.to_csv('stock_dfs/{}.csv'.format(ticker)))

https://en.xdnf.cn/q/118127.html

Related Q&A

sleekxmpp threaded authentication

so... I have a simple chat client like so:class ChatClient(sleekxmpp.ClientXMPP):def __init__(self, jid, password, server):sleekxmpp.ClientXMPP.__init__(self, jid, password, ssl=True)self.add_event_han…

DoxyPy - Member (variable) of namespace is not documented

I get the error message warning: Member constant1 (variable) of namespace <file_name> is not documented. for my doxygen (doxypy) documentation. I have documented the file and all functions and cl…

to_csv append mode is not appending to next new line

I have a csv called test.csv that looks like:accuracy threshold trainingLabels abc 0.506 15000 eew 18.12 15000And then a dataframe called summaryDF that looks like:accu…

Optional keys in string formats using % operator?

Is is possible to have optional keys in string formats using % operator? I’m using the logging API with Python 2.7, so I cant use Advanced String Formatting.My problem is as follow:>>> impor…

HDF5 headers missing in installation of netCDF4 module for Python

I have attempted to install the netCDF4 module several times now and I keep getting the same error:Traceback (most recent call last):File "<string>", line 17, in <module>File &quo…

How to catch network failures while invoking get() method through Selenium and Python?

I am using Chrome with selenium and the test run well, until suddenly internet/proxy connection is down, then browser.get(url) get me this:If I reload the page 99% it will load fine, what is the proper…

Pandas crosstab with own function

I have a function which takes two inputs and returns a float e.g. my_func(A, B) = 0.5. I have a list of possible inputs: x = [A, B, C, D, E, F].I want to produce a square matrix (in this case 6 by 6) …

Using a custom threshold value with tf.contrib.learn.DNNClassifier?

Im working on a binary classification problem and Im using the tf.contrib.learn.DNNClassifier class within TensorFlow. When invoking this estimator for only 2 classes, it uses a threshold value of 0.5 …

How to remove certain characters from a variable? (Python)

Lets suppose I have a variable called data. This data variable has all this data and I need to remove certain parts of it while keeping most of it. Lets say I needed to remove all the , (commas) in thi…

criticism this python code (crawler with threadpool)

how good this python code ? need criticism) there is a error in this code, some times script do print "ALL WAIT - CAN FINISH!" and freeze (no more actions are happend..) but i cant find reas…