How to properly compare a trading signal to a random strategy.

3 messages Options
Embed this post
Permalink
Harry Georgakopoulos

How to properly compare a trading signal to a random strategy.

Reply Threaded More More options
Print post
Permalink


 Set up ^^^^
 Let's say i have an evenly spaced discrete time-series of bid-ask prices.  Let's also say that the total number of such bid-ask pairs is N for a given day. Given a signal that generates a buy opportunity on "n" such times (where n << N), how can I  reliably say that these n signals have a mean profit that is statistically significant?
 For example, assume I get 35 buy signals throughout the day where I buy the offer, wait 5  minutes and then sell the bid.  This will generate a vector of 35 price-differences.  These price  differences will have a particular distribution.
 Thoughts  ^^^^^ 1.  I can compare the distribution of the 35 price-differences generated from the signal against      the distribution of 35 randomly chosen entry points throughout the day. (maybe some kind        of t-test on the difference of the means of these distributions)
 2.  I can compare the distribution of the 35 price-differences to a rolling window of all possible  buys throughout the day and selling after 5 mins.  (more data-points to compare against)
 3.  I can compare the distribution of the 35 price-differences against an absolute value of 0.
Any ideas on quantifying the significance of such a signal would be appreciated.  Is one method preferred over another?  Am I inadvertently introducing bias in the analysis?  I realize that the distribution of the price-differences might not be normally distributed.  This might make any analysis based on a t-test invalid.  
Thank you in advance.
H.


 
 
     
        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.
Brian G. Peterson

Re: How to properly compare a trading signal to a random strategy.

Reply Threaded More More options
Print post
Permalink
Harry Georgakopoulos wrote:

>
>  Set up ^^^^
>  Let's say i have an evenly spaced discrete time-series of bid-ask prices.  Let's also say that the total number of such bid-ask pairs is N for a given day. Given a signal that generates a buy opportunity on "n" such times (where n << N), how can I  reliably say that these n signals have a mean profit that is statistically significant?
>  For example, assume I get 35 buy signals throughout the day where I buy the offer, wait 5  minutes and then sell the bid.  This will generate a vector of 35 price-differences.  These price  differences will have a particular distribution.
>  Thoughts  ^^^^^
>  1.  I can compare the distribution of the 35 price-differences generated from the signal against      the distribution of 35 randomly chosen entry points throughout the day. (maybe some kind        of t-test on the difference of the means of these distributions)
>  2.  I can compare the distribution of the 35 price-differences to a rolling window of all possible  buys throughout the day and selling after 5 mins.  (more data-points to compare against)
>  3.  I can compare the distribution of the 35 price-differences against an absolute value of 0.
> Any ideas on quantifying the significance of such a signal would be appreciated.  Is one method preferred over another?  Am I inadvertently introducing bias in the analysis?  I realize that the distribution of the price-differences might not be normally distributed.  This might make any analysis based on a t-test invalid.  
> Thank you in advance.
> H.

Pat Burns has a paper on this topic on his website.

Regards,

     - Brian

--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.
David St John

Re: How to properly compare a trading signal to a random strategy.

Reply Threaded More More options
Print post
Permalink
In reply to this post by Harry Georgakopoulos
Harry,

My package 'ttrTests' is designed to do just that.  However, it doesn't
support bid-ask data.  It only uses a single price series for buys and
sells.  In the future I might be able to modify it to support bid/ask data.

Based on my 2 years of thesis research, the testing approach in the package
is what's prescribed in the literature.  You can go through my references
for more details.  The package documentation should explain briefly.  Even
though my package won't support your data, the methods for constructing test
statistics should all be valid.

http://cran.r-project.org/web/packages/ttrTests/ttrTests.pdf

Best,
-David

Date: Thu, 5 Nov 2009 20:29:09 -0600
From: Harry Georgakopoulos <[hidden email]>
Subject: [R-SIG-Finance] How to properly compare a trading signal to a
       random strategy.
To: <[hidden email]>
Message-ID: <[hidden email]>
Content-Type: text/plain



 Set up ^^^^
 Let's say i have an evenly spaced discrete time-series of bid-ask prices.
 Let's also say that the total number of such bid-ask pairs is N for a given
day. Given a signal that generates a buy opportunity on "n" such times
(where n << N), how can I  reliably say that these n signals have a mean
profit that is statistically significant?
 For example, assume I get 35 buy signals throughout the day where I buy the
offer, wait 5  minutes and then sell the bid.  This will generate a vector
of 35 price-differences.  These price  differences will have a particular
distribution.
 Thoughts  ^^^^^ 1.  I can compare the distribution of the 35
price-differences generated from the signal against      the distribution of
35 randomly chosen entry points throughout the day. (maybe some kind
 of t-test on the difference of the means of these distributions)
 2.  I can compare the distribution of the 35 price-differences to a rolling
window of all possible  buys throughout the day and selling after 5 mins.
 (more data-points to compare against)
 3.  I can compare the distribution of the 35 price-differences against an
absolute value of 0.
Any ideas on quantifying the significance of such a signal would be
appreciated.  Is one method preferred over another?  Am I inadvertently
introducing bias in the analysis?  I realize that the distribution of the
price-differences might not be normally distributed.  This might make any
analysis based on a t-test invalid.
Thank you in advance.
H.

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only.
-- If you want to post, subscribe first.