› ›

Demo Code : Read a File of Trades and Execute those Trades...

ptunneyptunney Posts: 246
edited January 2019 in How To

CLOUDQUANT ELITE ONLY

The code in this post can be found in SCRIPTS..PUBLIC..EXAMPLE..READ_AND_EXECUTE_TRADE_FILE_ELITE

A number of users would like to run a backtest that reads from a trades signal file : a file that contains a set of trading signals that they have already prepared, perhaps from a custom written script in another language or from a ML system.

This can be easily achieved using the following code. The script works with models run as a "Single MultiDay Job" so trades can straddle days.

We will use a file called signals.csv created and stored in your USER DATA tab on CloudQuant, which contains the following data.

Symbol,Timestamp,Qty,Side,Order_Type,Price
AAPL,20180710 09:15:01.200,200,buy,MOO,0
V,20180710 09:16:01.200,1000,sell,MOO,0
SPY,20180710 09:16:01.200,1000,buy,MOO,0
MSFT,20180710 09:45:04.200,600,buy,MKT,0
AAPL,20180710 09:55:02.300,100,buy,MKT,0
AAPL,20180710 12:20:02.300,200,sell,MKT,0
AAPL,20180710 15:29:00.000,500,buy,MKT,0
VHC,20180711 09:17:01.200,1000,sell,MOO,0
EEM,20180711 09:17:02.200,1000,sell,MOO,0
MSFT,20180711 10:05:05.200,100,sell,MKT,0
SPY,20180711 15:44:01.200,1000,sell,MOC,0
VHC,20180711 15:47:01.200,1000,buy,MOC,0
AAPL,20180712 12:20:02.300,500,sell,MKT,0
MSFT,20180712 13:50:00.000,200,sell,MKT,0
MSFT,20180713 09:30:00.001,300,sell,LMT,105.2
AAPL,20180713 15:30:35.495,100,sell,MOC,0
V,20180713 15:44:01.200,1000,buy,MOC,0
EEM,20180713 15:47:01.200,1000,buy,MOC,0

As a new feature we have added an UPLOAD button to the USER DATA tab to make it easier to get your own data files onto the system.

We read the data from the signals.csv using a single line of code. The return of this will be a list containing dictionary data.

datafile = service.read_file('signals.csv', 'csv')

To see what is in the "datafile" you can print it out by iterating through the list.

for my_trade in datafile:
    print (my_trade)

This produces the following prints:

{'Order_Type': 'MOO', 'Timestamp': '20180710 09:15:01.200', 'Price': 0.0, 'Side': 'buy', 'Qty': 200, 'Symbol': 'AAPL'}
{'Order_Type': 'MOO', 'Timestamp': '20180710 09:16:01.200', 'Price': 0.0, 'Side': 'sell', 'Qty': 1000, 'Symbol': 'V'}
{'Order_Type': 'MOO', 'Timestamp': '20180710 09:16:01.200', 'Price': 0.0, 'Side': 'buy', 'Qty': 1000, 'Symbol': 'SPY'}
{'Order_Type': 'MKT', 'Timestamp': '20180710 09:45:04.200', 'Price': 0.0, 'Side': 'buy', 'Qty': 600, 'Symbol': 'MSFT'}
{'Order_Type': 'MKT', 'Timestamp': '20180710 09:55:02.300', 'Price': 0.0, 'Side': 'buy', 'Qty': 100, 'Symbol': 'AAPL'}
{'Order_Type': 'MKT', 'Timestamp': '20180710 12:20:02.300', 'Price': 0.0, 'Side': 'sell', 'Qty': 200, 'Symbol': 'AAPL'}
{'Order_Type': 'MKT', 'Timestamp': '20180710 15:29:00.000', 'Price': 0.0, 'Side': 'buy', 'Qty': 500, 'Symbol': 'AAPL'}
{'Order_Type': 'MOO', 'Timestamp': '20180711 09:17:01.200', 'Price': 0.0, 'Side': 'sell', 'Qty': 1000, 'Symbol': 'VHC'}
{'Order_Type': 'MOO', 'Timestamp': '20180711 09:17:02.200', 'Price': 0.0, 'Side': 'sell', 'Qty': 1000, 'Symbol': 'EEM'}
{'Order_Type': 'MKT', 'Timestamp': '20180711 10:05:05.200', 'Price': 0.0, 'Side': 'sell', 'Qty': 100, 'Symbol': 'MSFT'}
{'Order_Type': 'MOC', 'Timestamp': '20180711 15:44:01.200', 'Price': 0.0, 'Side': 'sell', 'Qty': 1000, 'Symbol': 'SPY'}
{'Order_Type': 'MOC', 'Timestamp': '20180711 15:47:01.200', 'Price': 0.0, 'Side': 'buy', 'Qty': 1000, 'Symbol': 'VHC'}
{'Order_Type': 'MKT', 'Timestamp': '20180712 12:20:02.300', 'Price': 0.0, 'Side': 'sell', 'Qty': 500, 'Symbol': 'AAPL'}
{'Order_Type': 'MKT', 'Timestamp': '20180712 13:50:00.000', 'Price': 0.0, 'Side': 'sell', 'Qty': 200, 'Symbol': 'MSFT'}
{'Order_Type': 'LMT', 'Timestamp': '20180713 09:30:00.001', 'Price': 105.2, 'Side': 'sell', 'Qty': 300, 'Symbol': 'MSFT'}
{'Order_Type': 'MOC', 'Timestamp': '20180713 15:30:35.495', 'Price': 0.0, 'Side': 'sell', 'Qty': 100, 'Symbol': 'AAPL'}
{'Order_Type': 'MOC', 'Timestamp': '20180713 15:44:01.200', 'Price': 0.0, 'Side': 'buy', 'Qty': 1000, 'Symbol': 'V'}
{'Order_Type': 'MOC', 'Timestamp': '20180713 15:47:01.200', 'Price': 0.0, 'Side': 'buy', 'Qty': 1000, 'Symbol': 'EEM'}

Because this is a list of dictionaries you can now reference each of the elements by name. For example:

for my_trade in datafile:
    print(my_trade['Timestamp'])

Timestamps, datetime, MUTS - dealing with problems of converting your timestamp.

Trade files often have a string representation of a timestamp that looks something like this : 20180710 12:15:18.200

To process this we need to convert the timestamp string into a datetime field. Trades are often represented specifically in Eastern Standard Time. The backtesting system uses UTC time so we also have to take account of that. The following three lines of code take care of all of this.

        fileDt = datetime.strptime(my_trade['Timestamp'], '%Y%m%d %H:%M:%S.%f') # Convert the string to a datetime format
        fileDt = pytz.timezone('America/New_York').localize(fileDt) # Adjust input timestamps timezone NY (EST)
        dt_UTC = fileDt.astimezone(pytz.timezone('UTC')) #Get the UTC timestamp, because the simulator runs on UTC Time

You can change the first line to match the format of your own date/time data.

Full Working Script

# Replay a CSV File of Timestamped orders called signals.csv in 'User Data'. File columns = Symbol, Timestamp, Qty, Side, Order_Type, Price
# Example file contents for date period 2018/07/10 to 2018/07/13 at the bottom of this script, run 09:15 to 16:15 to allow MOO an MOC ordeers to execute.
# Timestamps in NY Time - 20180711 09:45:04.200, Side = "buy" or "sell", Order_Type = "MKT", "LMT", Quantity should be positive.
# Ensure your backtest runs for a time period covering all trades, especially for MOO MOC...
# PSEUDO CODE .....
# on_strategy_start - read the file, convert the date/time, select today's trades and store to a class dictionary
# is_symbol_qualified - qualify today's symbols
# on_start - set timers for all of today's trades
# on_timer - place trades

from cloudquant.interfaces import Strategy
from datetime import datetime
import pytz # Library solves timezone problems with UTC time

class Read_and_Execute_Trade_File_Timer_Version_ELITE(Strategy):

    @classmethod
    def on_strategy_start(cls, md, service, account):
        backtestDate = datetime.fromtimestamp(service.system_time/1000000) # get the date we are currently running in the simulator
        datafile = service.read_file('signals.csv', 'csv') # read the signal data file, which is in a CSV file with column labels.
        cls.todays_trades = [] # dictionary as a 'class' to contain all the signals from the file. on_strategy_start is only run once for single multiday backtests
        cls.symbol_list = [] # class variable to contain a list of the symbols we want to trade today    
        for my_trade in datafile: # step through each signal line in the file and store today's trades in a dictionary
            print("signal file read {}".format(my_trade))
            fileDt = datetime.strptime(my_trade['Timestamp'], '%Y%m%d %H:%M:%S.%f') # Convert the string to a datetime format
            if fileDt.date() == backtestDate.date(): # if the dates match we need to trade it today...
                fileDt = pytz.timezone('America/New_York').localize(fileDt) # Adjust input timestamps timezone NY (EST)
                dt_UTC = fileDt.astimezone(pytz.timezone('UTC')) #Get the UTC timestamp, because the simulator runs on UTC Time
                my_trade["NYtime"]=dt_UTC # add the converted time to the signal data
                cls.todays_trades.append(my_trade) # add the signal to the class dictionary of trades
                cls.symbol_list.append(my_trade['Symbol']) # the symbol to a list of symbols for today

    @classmethod
    def is_symbol_qualified(cls, symbol, md, service, account):
        return symbol in cls.symbol_list

    def on_start(self, md, order, service, account): #
        id = 0
        for my_trade in self.__class__.todays_trades:
            if my_trade['Symbol'] == self.symbol:
                print("adding trigger", my_trade['Symbol'],my_trade['Timestamp'],my_trade['Side'],my_trade['Order_Type'])
                dt = datetime.strptime(my_trade['Timestamp'], '%Y%m%d %H:%M:%S.%f').time()
                service.add_time_trigger(service.time(dt.hour, dt.minute, dt.second, dt.microsecond/1000),timer_id=id)
            id += 1

    def on_timer(self, event, md, order, service, account):
        TTrade =self.__class__.todays_trades[event.timer_id]
        print("Timer Triggered : ID {}, Timer Timestamp {}, File Timestamp {}".format(event.timer_id, service.time_to_string(event.timestamp),TTrade['Timestamp']))
        Side = TTrade["Side"]
        Order_Type = TTrade["Order_Type"]
        Price = TTrade["Price"]
        Qty = TTrade["Qty"]
        order.send(self.symbol, TTrade['Side'], TTrade['Qty'], type=TTrade['Order_Type'], price=TTrade['Price'])
        print("Order for {} sent at {}, Price {}, Side {}, Order Type {}, Quantity {}".format(self.symbol,service.time_to_string(event.timestamp),TTrade['Price'],TTrade['Side'],TTrade['Order_Type'],TTrade['Qty']))

    def on_fill(self, event, md, order, service, account): # on_fill will only be called for elite users running in normal mode (not fast)
        print ("{} Filled, Order Timestamp {}".format(event.symbol,service.time_to_string(event.timestamp)))


# Example file contents for trades from 2018/07/10 to 2018/07/13. Save into a file called signals.csv under 'User Data'
'''
Symbol,Timestamp,Qty,Side,Order_Type,Price
AAPL,20180710 09:15:01.200,200,buy,MOO,0
V,20180710 09:16:01.200,1000,sell,MOO,0
SPY,20180710 09:16:01.200,1000,buy,MOO,0
MSFT,20180710 09:45:04.200,600,buy,MKT,0
AAPL,20180710 09:55:02.300,100,buy,MKT,0
AAPL,20180710 12:20:02.300,200,sell,MKT,0
AAPL,20180710 15:29:00.000,500,buy,MKT,0
VHC,20180711 09:17:01.200,1000,sell,MOO,0
EEM,20180711 09:17:02.200,1000,sell,MOO,0
MSFT,20180711 10:05:05.200,100,sell,MKT,0
SPY,20180711 15:44:01.200,1000,sell,MOC,0
VHC,20180711 15:47:01.200,1000,buy,MOC,0
AAPL,20180712 12:20:02.300,500,sell,MKT,0
MSFT,20180712 13:50:00.000,200,sell,MKT,0
MSFT,20180713 09:30:00.001,300,sell,LMT,105.2
AAPL,20180713 15:30:35.495,100,sell,MOC,0
V,20180713 15:44:01.200,1000,buy,MOC,0
EEM,20180713 15:47:01.200,1000,buy,MOC,0
'''
Sign In or Register to comment.