Forex historical data in Python
Look for a watch, so be kind.
Solutions are needed to get historical Forex data in Python.
For stocks, it's easy:
import pandas as pd
import pandas_datareader as pdr
start = dt.date.today() - dt.timedelta(days=30)
end = dt.date.today()
df = pdr.DataReader('AAPL', 'google', start, end)
print(df.head())
Have tried Google, Yahoo, Fred and Oanda. Nothing works.
Please give an example code on how to request data. (In most cases, one line should be fine).
Thank.
source to share
Do you only need historical currency values?
Try using a module with a class (from a module ). I am using python 3, but I doubt this is too much. forex_python
datetime
datetime
These exchange rates are 3 hours (CET) data from the European Central Bank since 1999.
>>> from datetime import datetime
>>> from forex_python.converter import get_rate
>>> t = datetime(2001, 10, 18) # the 18th of October, 2001
>>> get_rate("USD", "GBP", t)
0.69233
>>> get_rate("GBP", "USD", t)
1.4444
>>> 1 / 1.4444 # check
0.6923289947382997
>>> t = datetime(2006, 6, 26) # June 26th, 2006
>>> get_rate("GBP", "USD", t)
1.8202
So
on 18/10/01, 1 USD = 0.69 GBP,
26 June 2006, 1 GBP == 1.82 USD.
source to share
The retail broker feed is always skewed, but I disagree that there are no good historical feeds. The industry standard for FX is the EBS channel . However, this is an expensive option. FXMarketAPI offers a channel that closely matches this. He is not affiliated with any broker. The API has a Pandas endpoint that helps you fetch data. Although there is a limit of 1000 requests for free users. You can see an example below.
URL = "https://fxmarketapi.com/apipandas"
params = {'currency' : 'EURUSD',
'start_date' : '2018-07-02',
'end_date':'2018-12-06',
'api_key':'**************'}
response = requests.get("https://fxmarketapi.com/apipandas", params=params)
df= pd.read_json(response.text)
source to share
Maybe you don't look hard enough :) A very nice looking guy posted this a few months ago. Admittedly, this is not the best code (the first open source project), however, it is currently under development and is constantly being improved. The next version will be much more efficient and cleaner.
fx_collect
Designed to store all FXCM historical data locally in Mariadb as follows.
+---------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+--------+
| date | bidopen | bidhigh | bidlow | bidclose | askopen | askhigh | asklow | askclose | volume |
+---------------------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+--------+
| 2017-04-27 10:01:00 | 17.294000 | 17.296000 | 17.289000 | 17.290000 | 17.340000 | 17.340000 | 17.334000 | 17.335000 | 113 |
| 2017-04-27 10:02:00 | 17.290000 | 17.298000 | 17.285000 | 17.295000 | 17.335000 | 17.342000 | 17.330000 | 17.340000 | 114 |
| 2017-04-27 10:03:00 | 17.295000 | 17.301000 | 17.289000 | 17.299000 | 17.340000 | 17.347000 | 17.340000 | 17.344000 | 98 |
| 2017-04-27 10:04:00 | 17.299000 | 17.300000 | 17.286000 | 17.295000 | 17.344000 | 17.345000 | 17.330000 | 17.340000 | 124 |
| 2017-04-27 10:05:00 | 17.295000 | 17.295000 | 17.285000 | 17.292000 | 17.340000 | 17.340000 | 17.330000 | 17.336000 | 130 |
| 2017-04-27 10:06:00 | 17.292000 | 17.292000 | 17.279000 | 17.292000 | 17.336000 | 17.336000 | 17.328000 | 17.332000 | 65 |
| 2017-04-27 10:07:00 | 17.292000 | 17.304000 | 17.287000 | 17.298000 | 17.332000 | 17.348000 | 17.332000 | 17.345000 | 144 |
| 2017-04-27 10:08:00 | 17.298000 | 17.306000 | 17.297000 | 17.302000 | 17.345000 | 17.350000 | 17.343000 | 17.346000 | 96 |
| 2017-04-27 10:09:00 | 17.302000 | 17.303000 | 17.294000 | 17.294000 | 17.346000 | 17.346000 | 17.338000 | 17.338000 | 50 |
| 2017-04-27 10:10:00 | 17.294000 | 17.296000 | 17.281000 | 17.291000 | 17.338000 | 17.338000 | 17.328000 | 17.333000 | 50 |
or if you just want basic tools to help you get started and create your own.
python-forexconnect
A demo account or Live FXCM is required to receive data. They provide free 10 year historical data on different time frames ( fxcm ).
source to share
FXCM recently released an official python wrapper for forexconnect.
There is a support forum: http://www.fxcodebase.com/code/viewforum.php?f=51&sid=e2b414c06f9714c605f117f74d689a9b
Here is a code snippet from the article on getting history:
from forexconnect import fxcorepy, ForexConnect
with ForexConnect() as fx:
try:
fx.login("user_id", "password", "fxcorporate.com/Hosts.jsp",
"Demo", session_status_callback=session_status_changed)
history = fx.get_history("EUR/USD", "H1",
datetime.datetime.strptime("MM.DD.YYYY HH:MM:SS", '%m.%d.%Y %H:%M:%S').replace(tzinfo=datetime.timezone.utc),
datetime.datetime.strptime("MM.DD.YYYY HH:MM:SS", '%m.%d.%Y %H:%M:%S').replace(tzinfo=datetime.timezone.utc))
source to share
There is free historical data based on pepper ticks in monthly CSV format starting in 2009 (visit: https://www.truefx.com/?page=downloads ) for most popular pairs. I wrote Python code using selenium to download all CSV files (the script will download all CSV files to the forex name folder):
import datetime, time, os
from dateutil.relativedelta import relativedelta
from selenium import webdriver
tmp_dir = os.path.join(os.getcwd(), 'forex')
if not os.path.isdir(tmp_dir): os.makedirs(tmp_dir)
options = webdriver.ChromeOptions();
options.add_argument("--window-size=1300,900")
options.add_experimental_option("prefs", {
"download.default_directory": tmp_dir,
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"safebrowsing.enabled": False,
"safebrowsing.disable_download_protection": True
})
options.add_argument("--disable-gpu")
options.add_argument("--disable-extensions")
options.add_argument('--disable-logging')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-certificate-errors-spki-list')
options.add_argument('--no-sandbox')
browser = webdriver.Chrome(options=options)
pairs = ['AUDJPY', 'AUDNZD', 'AUDUSD', 'CADJPY', 'CHFJPY', 'EURCHF', 'EURGBP', 'EURJPY', 'EURUSD', 'GBPJPY', 'GBPUSD', 'NZDUSD', 'USDCAD', 'USDCHF', 'USDJPY']
for pair in pairs:
curr_date = datetime.datetime(2015, 1, 1)
while curr_date + relativedelta(months=1) < datetime.datetime.now():
file_name = '{p}-{ym2}.zip'.format(p=pair, ym2=curr_date.strftime('%Y-%m'))
url = 'http://www.truefx.com/dev/data/{y}/{ym1}/{p}-{ym2}.zip'.format(
y=curr_date.strftime('%Y'),
ym1=curr_date.strftime('%B').upper()+'-'+curr_date.strftime('%Y') if curr_date <= datetime.datetime(2017, 3, 1) else curr_date.strftime('%Y-%m'),
p=pair,
ym2=curr_date.strftime('%Y-%m')
)
file_found = False
for root, dirs, files in os.walk(tmp_dir):
for file in files:
if file_name in file: file_found = True
if not file_found:
time.sleep(5)
browser.get (url)
file_downloaded = False
while not file_downloaded:
time.sleep(1)
for root, dirs, files in os.walk(tmp_dir):
for file in files:
if file_name in file and not '.crdownload' in file: file_downloaded = True
print(file_name, 'downloaded from', url)
curr_date = curr_date + relativedelta(months=1)
print('completed')
source: https://gist.github.com/mamedshahmaliyev/bca9242b7ea6a13b3f76dee7a5aa111a
source to share
You could use fxcmpy. http://fxcmpy.tpq.io/
here's a quick example:
import matplotlib.pyplot as plt
import datetime as dt
import fxcmpy
con = fxcmpy.fxcmpy(config_file='fxcm.cfg')
# must optain API Token, see link for details.
start = dt.datetime(2018, 7, 6,8,0,0)
end = dt.datetime(2018, 7, 7,18,0,0)
c = con.get_candles('XAU/USD', period='m1', columns=['bidclose','tickqty'], start=start, end=end )
# Basic plotting of close and volumne data
fig, ax = plt.subplots(figsize=(11,8))
ax.plot(c.index,c['bidclose'], lw=1, color='B',label="Close")
ax2= ax.twinx()
ax2.plot(c.index,c['tickqty'], lw=1, color='G',label="Volume")
plot.show()
source to share
(cit. :) Most of the time one line should be ok?
This cannot be more mistaken.
There is nothing like historical FOREX data. Each FX reseller (Broker) creates its own trading conditions. Even the same Broker can provide several different (or inconsistent if he wishes) price feeds for the same currency pair trade so that every T&C "product" can be executed.
The FOREX ecosystem is a decentralized, multi-agent / multi-processor, predominantly distributed global marketplace.
Rather, forget to have the SLOC, the magic one-line, to get a universally valid response from some nonexistent divine API. There is no such.
Yes, it can receive FX data - but each Broker provides a different image:
Yes, it is possible to integrate the localhost process with a separate API service from one specific Broker for one specific trading account (see the relevant T&C for the detailed context of such data transfer).
Some brokers publish their local tick data and some do not. Some research agencies may be able to assist you in some of your research efforts and share selected tick data segments for a specific CCY pair. But there is zero global consolidation. There is simply no reason to aggregate such a service that has zero value added.
If in vitro quantitative modeling alone is to make any sense, the model must be validated against the same market where in-vivo trade is expected to take place.
So you need one specific market access information (Broker asked for this) where your service goes to work in-vivo.
source to share