Social Sentiment from SMA (Alternative Data) - Making a losing strategy profitable

td753764td753764 Posts: 67

Hi All,

Over the past months, I have spent some time working on our alternative data. There is a lot more to come soon.

However, I did take some time to put together a backtest that shows some of the power of this alternative data.

Alternative datasets are improving our trading strategies at CloudQuant. In the following example, we added an easy check of social sentiment. Before going long, check to see if social sentiment was favorable. Instead of following the original algo, simply add this additional check. If sentiment was positive, go ahead and go long. Otherwise, don’t trade. This social sentiment provided an insurance policy essentially for the underlying strategy, and it improves the performance. One can easily see the results in the following diagrams where the first trading strategy lost money and the second made over $170,000.

Original Breakout Strategy Without Alternative Data

Same Script - Adding in Check for SMA Alternative Data

Tagged:

Comments

  • --- Note -- This script works for CQ Elite...
    to request upgrade to elite use: https://forum.cloudquant.com/discussion/206/upgrading-to-cq-elite#latest

  • Source Code to the script with the SMA Sentiment

    # Created by Tayloe Draughon - Product Manager, CloudQuant
    # This is a strategy designed to follow major market movements (up or down). 
    # It enters orders to put on a position when a given stocks is trading
    # well outside their average from previous days.
    
    # This version of the script uses SMA to validate that we should
    # enter into a position.
    
    from cloudquant.interfaces import Strategy
    from cloudquant.util import dt_from_muts
    import numpy as np
    ##########################################
    # Global Variables for the script
    from cloudquant.interfaces import Strategy
    from cloudquant.util import dt_from_muts
    import numpy as np
    
    
    end_delay = 5 # in minutes, how long before the end of the day to stop trading.
    start_delay = 10 # in minutes, how long after market open before we start trading
    index=5 # how many days of highs to average over
    purchase_atr_ratio=3.0 # what fraction of the atr for a stock to be bought/shorted
    sell_atr_ratio=3.0 # what fraction of the atr for a stock to be exited from
    filename = "breakout_sma.csv"  #user file where data is collected.
    
    
    allow_shorts = False  # if set to True, we will allow short sales on downward breakouts
    MAX_PNL_PER_SHARE = 1.75 # Target profit or loss for the strategy
    MAX_PARTICIPATION = 0.03 #Maximum percentage of the previous minuteBar volume that we will trade.
    MIN_ORD_QTY = 10 # fewest number of shares that we will order when entering into a position.
    MAX_ORD_QTY = 800 #maximum number of shares that we will order when entering into a position.
    
    
    class breakout_withSMA(Strategy):
        @classmethod
        def is_symbol_qualified(cls, symbol, md, service, account):
            handle_list = service.symbol_list.get_handle('9a802d98-a2d7-4326-af64-cea18f8b5d61') #this is all stocks on S&P500
            return service.symbol_list.in_list(handle_list,symbol)
            #return symbol in ['AAPL', 'EBAY', 'AMZN', 'ORCL', 'WMT']
    
        #############################################################################
        # Setting up to get information from SMA for our s-score on social sentiment
        #############################################################################
        @classmethod
        def register_event_streams(cls, md, service, account):
            #return {'!sentiment/sma/tw/15min': 'on_event_sma_tw', '!sentiment/sma/st/15min':'on_event_sma_tw'}
            return {'!sentiment/sma/st/15min': 'on_event_sma_st', '!sentiment/sma/st/15min':'on_event_sma_st'}
    
        def on_event_sma_st(self, event, md, order, service, account):
            #print service.time_to_string(event.timestamp),"sma_st",event.field['s-score']
            self.sma_s_score = event.field['s-score']
            #print event
    
        def on_event_sma_tw(self, event, md, order, service, account):
            #print service.time_to_string(event.timestamp),"sma_tw",event.field['s-score']
            self.sma_s_score = event.field['s-score']
            #print event        
    
    
        @classmethod
        def on_strategy_start(cls,md, service,account):
            title_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
            service.write_file( filename, title_row )
    
        def __init__(self):  
    
            self.entry_price = 0  # estimated price of our position
            self.model_start = 0 # time to start, set in on_start
            self.IsPurchasable=True # OK to purchase? (not repurchasing what we already sold)
            self.OrdQty = MAX_ORD_QTY # This variable will change with the volume of the previous bar * MAX_PARTICIPATION
            self.sma_s_score = 0
            self.sma_s_score_at_Entry = 0
            self.IsInClose = False
            self.PosToday = False
            self.ClosingOrder = -1
    
            ############################################################
            # daily bar statistics populated properly in strategy start
            self.md_daily=0
            self.md_high=0
            self.average_high=0
            self.md_low=0
            self.average_low=0
            self.target_profit = 1.0
    
    
        def on_finish(self, md, order, service, account):
            pass
    
        def on_minute_bar(self, event, md, order, service, account, bar):        
            bar_1 = bar.minute()
            bar_close = bar_1.close
            try:
                self.OrdQty = int(bar_1.volume[0]* MAX_PARTICIPATION)
                if self.OrdQty   >  MAX_ORD_QTY:
                    self.OrdQty  = MAX_ORD_QTY
                if self.OrdQty <= 0:
                    self.OrdQty = MIN_ORD_QTY
            except:
                self.OrdQty = MIN_ORD_QTY
            ###############################################################
            # make sure it's not too late in the day - We don't want to open
            # a position in the closing period of the day.
            if service.system_time < md.market_close_time - service.time_interval(minutes=end_delay, seconds=1):
                bar_1 = bar.minute()
                bar_close = bar_1.close
    
                # no need to calc alpha signal if there is a position on already
                if account[self.symbol].position.shares == 0:
                    # get the 1 minute bar data
                    if len(bar_close) > 0 and bar_close[0] != 0:
                        #if the stock has returned to its normal values, we would consider entering a position on it.
                        if (self.average_high+purchase_atr_ratio * md.stat.atr) > bar_close[0] \
                            and (self.average_low-purchase_atr_ratio * md.stat.atr) < bar_close[0]:
                            if self.PosToday != True:  ############## Only allow one position per day per symbol
                                try:
                                    if bar_1.volume[0] < 5 * MAX_ORD_QTY:
                                        self.IsPurchasable=False ### Too Thinly Traded
                                    else: 
                                        self.IsPurchasable=True
                                except:
                                    self.IsPurchasable=False
    
                        # we want to have at least a certain amount of time left before entering positions
    
                        if self.IsPurchasable==True and self.PosToday != True:  ############## No position on, therefore if purchasable, enter into a position.
    
                            # go long if stock is well above its normal values
                            if (self.average_high+purchase_atr_ratio*md.stat.atr)<bar_close[0]:
                                #print(self.symbol)
                                #print(self.average_high)
                                #print(md.stat.atr)
                                #print(bar_close[0])
                                if self.sma_s_score >= 0.5:
                                    self.PosToday = True
                                    #print("symbol {}, Volume = {} OrdQty = {}".format(self.symbol, bar_1.volume[0], self.OrdQty))
                                    print('{}\tBUY\t{}\tQty: {}\tBreakout\tPX: {}\tS-Score: {}'\
                                      .format(service.time_to_string(service.system_time), self.symbol, self.OrdQty,  bar_close[0], self.sma_s_score))
                                    order_id = order.twap(self.symbol, self.OrdQty, "buy", time_frame=1,order_aggression=3)
                                    self.entry_price = bar_close[0]
                                    self.sma_s_score_at_Entry = self.sma_s_score
    
                                    #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                                    data_row ="{}\t{}\tbuy\t{}\t\t{}\t\tToOpen\t{}\t{}\t{}\t{}".format(\
                                        service.time_to_string(service.system_time),\
                                        self.symbol,\
                                        self.entry_price,\
                                        self.sma_s_score_at_Entry,\
                                        self.average_high,\
                                        self.average_low,\
                                        self.md_high,\
                                        self.md_low)
                                    service.write_file( filename, data_row )
    
    
                            # short if the stock is well below its normal values
                            elif (self.average_low-purchase_atr_ratio*md.stat.atr)>bar_close[0] and allow_shorts == True:
                                #print(self.symbol)
                                #print(self.average_high)
                                #print(md.stat.atr)
                                #print(bar_close[0])
                                if self.sma_s_score <= -0.5:
                                    self.PosToday = True
                                    print('{}\tSSRT\tQty: {}\t{}\tBreakout\tPX: {}\tS-Score: {}'\
                                      .format(service.time_to_string(service.system_time), self.symbol, self.OrdQty, bar_close[0], self.sma_s_score))
                                    order_id = order.twap(self.symbol, self.OrdQty, "sell", time_frame=1,order_aggression=3)
                                    self.entry_price = bar_close[0]
                                    self.sma_s_score_at_Entry = self.sma_s_score
    
                                    #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                                    data_row ="{}\t{}\tSSRT\t{}\t\t{}\t\tToOpen\t{}\t{}\t{}\t{}".format(\
                                        service.time_to_string(service.system_time),\
                                        self.symbol,\
                                        self.entry_price,\
                                        self.sma_s_score_at_Entry,\
                                        self.average_high,\
                                        self.average_low,\
                                        self.md_high,\
                                        self.md_low)
                                    service.write_file( filename, data_row )
    
    
                else:  ########## position isn't zero
                    ###########################################################
                    # Are we long? and without an already placed closing order
                    if account[self.symbol].position.shares > 0 and self.ClosingOrder < 0:
                        # there is a position on, therefore we want to check to see if
                        # we should realize a profit or stop a loss
                        self.IsPurchasable=False
                        try:
                            if bar_close[0] < (account[self.symbol].position.entry_price - self.target_profit):
                                #### stop my loss now.
                                #self.ClosingOrder = order.send(self.symbol, 'sell', account[self.symbol].position.shares, type='MKT')
                                self.ClosingOrder = order.twap(self.symbol,  abs(account[self.symbol].position.shares), "sell", time_frame=1,order_aggression=2)
                                print('{}\tSell\t{}\tStop To Close\tEntryPx: {}\tExitPx: {}\tP&L: {}\tTarget P&L: {}\tEOD\tEntrySscore: {}\tCloseSscore: {}'\
                                   .format(service.time_to_string(service.system_time), self.symbol, account[self.symbol].position.entry_price, bar_close[0], 
                                          (bar_close[0] - account[self.symbol].position.entry_price), self.target_profit, self.sma_s_score_at_Entry, self.sma_s_score))
                                #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                                data_row ="{}\t{}\tSell\t{}\t{}\t{}\t\tStopLoss\t{}\t{}\t{}\t{}".format(\
                                    service.time_to_string(service.system_time),\
                                    self.symbol,\
                                    self.entry_price,\
                                    bar_close[0],\
                                    self.sma_s_score_at_Entry,\
                                    self.average_high,\
                                    self.average_low,\
                                    self.md_high,\
                                    self.md_low)
                                service.write_file( filename, data_row )
                            elif bar_close[0] > (account[self.symbol].position.entry_price + self.target_profit):
                                #### take profit now.
                                #self.ClosingOrder = order.send(self.symbol, 'sell', account[self.symbol].position.shares, type='MKT')
                                self.ClosingOrder = order.twap(self.symbol,  abs(account[self.symbol].position.shares), "sell", time_frame=1,order_aggression=2)
                                print('{}\tSell\t{}\tTo Close\tEntryPx: {}\tExitPx: {}\tP&L: {}\tTarget P&L: {}\tEOD\tEntrySscore: {}\tCloseSscore: {}'\
                                   .format(service.time_to_string(service.system_time), self.symbol, account[self.symbol].position.entry_price, bar_close[0], 
                                          (bar_close[0] - account[self.symbol].position.entry_price),self.target_profit, self.sma_s_score_at_Entry, self.sma_s_score))
                                #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                                data_row ="{}\t{}\tSell\t{}\t{}\t{}\t\tTakeProfit\t{}\t{}\t{}\t{}".format(\
                                    service.time_to_string(service.system_time),\
                                    self.symbol,\
                                    self.entry_price,\
                                    bar_close[0],\
                                    self.sma_s_score_at_Entry,\
                                    self.average_high,\
                                    self.average_low,\
                                    self.md_high,\
                                    self.md_low)
                                service.write_file( filename, data_row )
                        except:
                            pass
    
                    ############################
                    # Are we short? and without an already placed
                    elif account[self.symbol].position.shares < 0  and self.ClosingOrder < 0: 
                        # there is a position on, therefore we want to check to see if
                        # we should realize a profit or stop a loss
                        self.IsPurchasable=False
                        try:
                            if bar_close[0] > (account[self.symbol].position.entry_price + 1.0):
                                #### stop my loss now.
                                #self.ClosingOrder = order.send(self.symbol, 'buy', account[self.symbol].position.shares, type='MKT')
                                self.ClosingOrder = order.twap(self.symbol,  abs(account[self.symbol].position.shares), "buy", time_frame=1,order_aggression=2)
                                print('{}\tBuy\t{}\tStop To Close\tEntryPx: {}\tExitPx: {}\tP&L: {}\tTarget P&L: {}\tEOD\tEntrySscore: {}\tCloseSscore: {}'\
                                   .format(service.time_to_string(service.system_time), self.symbol, account[self.symbol].position.entry_price, bar_close[0], 
                                          ( account[self.symbol].position.entry_price - bar_close[0]), self.target_profit, self.sma_s_score_at_Entry, self.sma_s_score))
    
                                #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                                data_row ="{}\t{}\tBuy\t{}\t{}\t{}\t\tStopLoss\t{}\t{}\t{}\t{}".format(\
                                    service.time_to_string(service.system_time),\
                                    self.symbol, self.entry_price,\
                                    bar_close[0], self.sma_s_score_at_Entry,\
                                    self.average_high, self.average_low,\
                                    self.md_high, self.md_low)
                                service.write_file( filename, data_row )
                            elif bar_close[0] < (account[self.symbol].position.entry_price - 1.0):
                                #### take profit now
                                #self.ClosingOrder = order.send(self.symbol, 'buy', account[self.symbol].position.shares, type='MKT')
                                self.ClosingOrder = order.twap(self.symbol,  abs(account[self.symbol].position.shares), "buy", time_frame=1,order_aggression=2)
                                print('{}\tBuy\t{}\tTo Close\tEntryPx: {}\tExitPx: {}\tP&L: {}\tTarget P&L: {}\tEOD\tEntrySscore: {}\tCloseSscore: {}'\
                                   .format(service.time_to_string(service.system_time), self.symbol, account[self.symbol].position.entry_price, bar_close[0], 
                                          ( account[self.symbol].position.entry_price - bar_close[0]), self.target_profit, self.sma_s_score_at_Entry, self.sma_s_score))
    
                                #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                                data_row ="{}\t{}\tBuy\t{}\t{}\t{}\t\tTakeProfit\t{}\t{}\t{}\t{}".format(\
                                    service.time_to_string(service.system_time),\
                                    self.symbol, self.entry_price,\
                                    bar_close[0], self.sma_s_score_at_Entry,\
                                    self.average_high, self.average_low,\
                                    self.md_high, self.md_low)
                                service.write_file( filename, data_row )
                        except:
                            pass
    
    
            else:
                # close out of our positions at the end of the day 
                if self.IsInClose == False:  # Only want to do the closing logic once.
                    self.IsInClose = True
                    bar_1 = bar.minute()
                    bar_close = bar_1.close
                    if account[self.symbol].position.shares > 0:
                        self.ClosingOrder = order.twap(self.symbol,  abs(account[self.symbol].position.shares), "sell", order_aggression=3)
                        if len(bar_close) > 0:
                            print('{}\tSell\t{}\tTo Close\tEntryPx: {}\tExitPx: {}\tP&L: {}\tTarget P&L: {}\tEOD\tEntrySscore: {}\tCloseSscore: {}'\
                                  .format(service.time_to_string(service.system_time), self.symbol, account[self.symbol].position.entry_price, bar_close[0], 
                                          (bar_close[0] - account[self.symbol].position.entry_price), self.target_profit, self.sma_s_score_at_Entry, self.sma_s_score))
                            #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                            data_row ="{}\t{}\tSell\t{}\t{}\t{}\t{}\tToClose\t{}\t{}\t{}\t{}".format(\
                                            service.time_to_string(service.system_time),\
                                            self.symbol,\
                                            self.entry_price,\
                                            bar_close[0], \
                                            self.sma_s_score_at_Entry,\
                                            self.sma_s_score, \
                                            self.average_high,\
                                            self.average_low,\
                                            self.md_high,\
                                            self.md_low)
                            service.write_file( filename, data_row )
    
                        else:
                            print('{}\tSell\t{}\tTo Close\tEntryPx: {}\tExitPx: {}\t\tEntrySscore: {}\tCloseSscore: {}'\
                                  .format(service.time_to_string(service.system_time), self.symbol,account[self.symbol].position.entry_price,"unknown", self.sma_s_score_at_Entry, self.sma_s_score))
                            #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                            data_row ="{}\t{}\tSell\t{}\t\t{}\t{}\tToClose\t{}\t{}\t{}\t{}".format(\
                                            service.time_to_string(service.system_time),\
                                            self.symbol,\
                                            self.entry_price,\
                                            self.sma_s_score_at_Entry,\
                                            self.sma_s_score, \
                                            self.average_high,\
                                            self.average_low,\
                                            self.md_high,\
                                            self.md_low)
                            service.write_file( filename, data_row )
    
    
                    # close out of our short positions at the end of the day
                    if account[self.symbol].position.shares < 0 :
                        self.ClosingOrder = order.twap(self.symbol,  abs(account[self.symbol].position.shares), "sell", order_aggression=3)
                        if len(bar_close) > 0:
                            print('{}\tBuy\t{}\tTo Close\tEntryPx: {}\tExitPx: {}\tP&L: {}\tTarget P&L: {}\tEOD\tEntrySscore: {}\tCloseSscore: {}'\
                                  .format(service.time_to_string(service.system_time), self.symbol,account[self.symbol].position.entry_price, bar_close[0], 
                                          (account[self.symbol].position.entry_price-bar_close[0]), self.target_profit, self.sma_s_score_at_Entry, self.sma_s_score))
                            #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                            data_row ="{}\t{}\tBuy\t{}\t{}\t{}\t{}\tToClose\t{}\t{}\t{}\t{}".format(\
                                            service.time_to_string(service.system_time),\
                                            self.symbol,\
                                            self.entry_price,\
                                            bar_close[0], \
                                            self.sma_s_score_at_Entry,\
                                            self.sma_s_score, \
                                            self.average_high,\
                                            self.average_low,\
                                            self.md_high,\
                                            self.md_low)
                            service.write_file( filename, data_row )
    
                        else:
                            print('{}\tBuy\t{}\tTo Close\tEntryPx: {}\tExitPx: {}\t\tEntrySscore: {}\tCloseSscore: {}'\
                                  .format(service.time_to_string(service.system_time), self.symbol,account[self.symbol].position.entry_price,"unknown", self.sma_s_score_at_Entry, self.sma_s_score))
                            #data_row ="Timestamp\tSymbol\tB-S\tEntryPx\tExitPx\tEntryS-Score\tExitS-Score\tAction\tDailyAvgHigh\tDailyAvgLow\tDailyBarHigh\tDailyBarLow"
                            data_row ="{}\t{}\tBuy\t{}\t{}\t\t{}\tToClose\t{}\t{}\t{}\t{}".format(\
                                            service.time_to_string(service.system_time),\
                                            self.symbol,\
                                            self.entry_price,\
                                            self.sma_s_score_at_Entry,\
                                            self.sma_s_score, \
                                            self.average_high,\
                                            self.average_low,\
                                            self.md_high,\
                                            self.md_low)
                            service.write_file( filename, data_row )
    
    
        def on_start(self, md, order, service, account):
            self.model_start = service.system_time + service.time_interval(minutes=start_delay, seconds=1)
            #gather some statistics
            self.md_daily=md.bar.daily(start=-index)
            self.md_high=self.md_daily.high
            self.average_high=np.mean(self.md_daily.high)
            self.md_low=self.md_daily.low
            self.average_low=np.mean(self.md_daily.low)
    
            self.target_profit = self.average_high - self.average_low
    
            if self.target_profit > MAX_PNL_PER_SHARE:
                self.target_profit = MAX_PNL_PER_SHARE
    
    
    
    
            pass
    
  • Hi mr
    How can I get your database?

  • ptunneyptunney Posts: 246

    This dataset is now only available as a paid dataset from https://catalog.cloudquant.com

Sign In or Register to comment.