nkaz001 / hftbacktest Goto Github PK

Hi! Thank you for this awesome project. I have a remark regarding the rebates:

After backtesting a strategy using this framework, I decided to test it live using a minimal investment. I collected the live OB and trade data to compare the results of the live trading session with what the backtesting framework would output.
After an aprox 12 hour session, I went from 59.13 dollars to 55.89.

I then ran the backteting framework on the same interval with the same data. Everything coincides with the live session (my fair price, half spread, skew, inventory position are all roughly the same). The number of trades and the volume is similar with the live trading session as well. What doesn't coincide is the actual PnL. The backtest framework went from 59.13 to 58.64.

I attached the backtest PnL plot:

The summary:

=========== Summary ===========
Sharpe ratio: -8.6
Sortino ratio: -8.9
Risk return ratio: -244.9
Annualised return: -858.60 %
Max. draw down: 3.51 %
The number of trades per day: 461
Avg. daily trading volume: 0
Avg. daily trading amount: 20021
Max leverage: 8.09
Median leverage: 0.74

I use Bitmex fees, which are -0.015% for linear contracts. All my orders are GTX only. Order size is always the minimum of 0.001 BTC.
My HftBacktest call is:

    hbt = HftBacktest(
        [data[-1]],
        tick_size=0.5,
        lot_size=0.001,
        maker_fee=-0.015 / 100,
        taker_fee=0.075 / 100,
        order_latency=ConstantLatency(entry_latency=50000, response_latency=25000),
        queue_model=SquareProbQueueModel(),
        asset_type=Linear,
        trade_list_size=100_00000,
        exchange_model=PartialFillExchange
    )

Ok, so the rebate is 0.015% and the volume is 20021, so the total fees paid back to me are 20021 * 0.015 / 100 = 3.00315. As you can see from the summary, the return excluding fees is aprox -11%, which leaves us with 52.777 at the end of the trading session. Adding up the fees we end up with 55.78, which is very close to my live results, but way less then the backtest results.
Using twice the rebate, we end up with aprox 58.7833, which coincides with the backtest result.

Am I misunderstanding something or is the rebate calculation flawed? (If needed, I can send you an email with my code + data.)

For the orderbook in snapshot format, we know that we need to convert it to incremental format to put the data in.

For the stock market, the day starts at 9:00 and the regular day ends at 4:30 pm. In this case, can I ask for some guidelines on how to handle DEPTH_CLEAR_EVENT and DEPTH_SNAPSHOT_EVENT?

Also, I appreciate your open source, and would like to support you, either as a github sponsor or on Patreon would be great.

Dear nkaz,

First of all, I'd like to thank you very much for this amazing repository. I feels good to find someone who`s trying to implement the same papers that I've been reading over the last few years and also trying to implement myself.

I still haven`t been able to successfully implement those papers in production, and I am really (really) looking forward to it. I would to leave some comments, maybe we can find a solution together.

First remark, on the effect of kappa on the bid and ask quotes. On "Dealing with the Inventory Risk", section 6.5, it is discussed that kappa impacts the bid-ask spread. However, as pointed out by Lehalle on stack exchange, there are two reference papers that describe how to calculate k and A. They are Tapia's thesis and Sophie Laruelle's "Faisabilité de l’apprentissage des paramètres d’un algorithme de trading sur des données réelles". Even though it's in French, it is possible to see on section 3.5 that she describes kappas and As for bid and ask separately.

Suppose bid kappa is very high, and ask kappa is very low. This means that the bid will be offset to a much lower price, while the ask will be offset towards the mid price. The concept of half spread doesn't exist anymore, in the sense that the spread is not symmetric even if the inventory is 0. From my interpretation, it's an implicit adjustment to the market conditions, meaning that the demand for bids is higher than the demand for asks (market is likely in a downtrend). The market maker is more likely to be losing money is such conditions, if he assumes equal kappas for both sides.

The seconds remark is regarding the multi asset solution utilizing Gueant's framework. You are skipping the part where you have to compute the correlation matrix of the asset's in your portfolio, so your "inventory" doesn't have to be an inventory in terms of quantities (even though it sounds a good idea to hardcode raw inventory limits) but in terms of delta in respect to your reference asset.

Suppose a market making portfolio of ETHUSDT and BTCUSDT, and the correlation between ETHBTC is 0.8. If you are long 80 BTC, and short 100 ETH, your delta is basically 0 and you can continue quoting without necessarily adjusting your quotes to zero your inventories, as discussed in item 6 of Gueant's "Optimal Market Making".

From my personal experience, the hardest part of making money with theses models is in regards to being forced to send adjusted quotes when the market is against you (you may very well lose your profits and realize a loss) and the second difficulty is in regards to the velocity at which you accumulate inventory that goes against you. You have hardcoded adj2 to circumvent that, but still will be losing excl fees.

My user is sandstorm111#2242 on discord, if you feel like contacting me directly.

Kind regards

In example.py, snapshot_df uses 20220830, whilst df uses 20220831.
Why do they use two different dates?
Thanks.

Hello,

I have some data which is trades data, and full order snapshots only. I've created a import and digest strategy based off of Tardis and BinanceFutures import regime.

Will the backtest function without DEPTH_EVENTs eg I only have DEPTH_CLEAR_EVENT, DEPTH_SNAPSHOT_EVENT and TRADE_EVENT? Or am I barking up a wrong tree here?

Everything seems to load in and the data structure and aggregation match what appears to be the desired output and example functions run, but what the backtester sees and what the data is aren't congruent.

Trades:

Snapshot of 5 levels (which are consumed and produced into multiple rows with same execution timestamp and DEPTH_CLEAR_EVENT before DEPTH_SNAPSHOT_EVENT)

When digested and combined the resulting dataset:

Example output from the data aggregation (and following the Data Preparation workbook)

data = create_last_snapshot(f"{DATA_FOLDER}{EXPORT_FILE_NAME}", tick_size=TICK_SIZE, lot_size=LOT_SIZE)
# Create snapshot for end of day.
np.savez(f"{DATA_FOLDER}{EXPORT_SNAPSHOT_FILE_NAME}", data=data)
data
array([[ 4.0000000e+00,  1.6833258e+15, -1.0000000e+00,  1.0000000e+00,
         2.0612000e+03,  1.0000000e+00],
       [ 4.0000000e+00,  1.6833258e+15, -1.0000000e+00,  1.0000000e+00,
         2.0611000e+03,  9.0000000e+00],
       [ 4.0000000e+00,  1.6833258e+15, -1.0000000e+00,  1.0000000e+00,
         2.0610000e+03,  8.0000000e+00],
       ...,
       [ 4.0000000e+00,  1.6833258e+15, -1.0000000e+00, -1.0000000e+00,
         2.0615000e+03,  1.5000000e+01],
       [ 4.0000000e+00,  1.6833258e+15, -1.0000000e+00, -1.0000000e+00,
         2.0616000e+03,  6.0000000e+00],
       [ 4.0000000e+00,  1.6833258e+15, -1.0000000e+00, -1.0000000e+00,
         2.0617000e+03,  1.1000000e+01]])

Happy to provide more details and additional information, just wondering where I might start (as the only key difference is the lack of DEPTH_EVENTs when comparing the two datasets).

I want to backtest in a wide range of time series, so I need to use Binance's open public data(https://github.com/binance/binance-public-data). However, I am not sure about the exact data structure in hftbacktest. Can you provide guidelines or an example code for importing external data?

I wanted to take advantage of the freely available historical futures orderbook level 2 data from binance.

It should be possible by combining this with historical trade data (also available from binance I believe) to obtain normalized data for htfbacktest.

But I couldn't find this in the repo examples. I wanted to check if it has already been done, so I don't waste time redoing it?

I built hftbacktest from source with python -m build, and then installed it.

When I then tried the Getting Started notebook, I got the following error:

ImportError: cannot import name 'randomdecl' from 'numba.core.typing' 

After uninstalling, and reinstalling using pip install hftbacktest the problem is gone.

I thought I should point this out.

Tardis incremental_book_l2_*.csv.gz files contain a snapshot at the very beginning of each file.

In the doc it is nicely explained how to create a snapshot EOD, which is then passed to the backtest function.

I assume that this is not required for Tardis data, however it is not clear as the doc also says:

You can also build the snapshot in the same way as described above.

Can you clarify if this is needed. I also checked the code and it seems not needed.
Thanks for clarifying this, and eventually update the doc. If it is needed we might need a function to extract the beginning of day snapshot from a Tardis incremental_book_l2 file?

From Calculating Trading Intensity:

# All of our possible quotes within the order arrival depth,
# excluding those at the same price, are considered executed.
out[:tick] += 1

When calculating trading intensity, the trades are aggregated down to the mid. Is it really a fair thing to do? In there was a big order smashing throught multiple levels, we’ll receive all corresponding trades from the data feed. Also there may be not much liquidity around the mid (wide bbo spread), but this way to model it will tell us that most of trading was half tick away from the mid.

I tried to collect the trades without aggregating them down to the mid, and the resulting intensity chart is not a nice decaying exponential curve. To the point, that it’s almost unfittable.

Hi @nkaz001!
I am very interested in the great concept and implementation of hftbacktest and currently working on a tutorial on the Guéant-Lehalle-Fernandez-Tapia Market Making Model and Grid Trading.
I am implementing it using Tardis data at 5/31 2023 for snapshot and at 6/1 2023 for backtesting, but hbt.last_trades is always an empty list and unable to estimate the intensity function.
The data is created by tardis.convert and there is an abundance of data with event==2 at 6/1 2023 as shown below.

I tried to find the cause by printing self.last_trades, but they all seem to have a length of 0.
Below is the code.

@njit
def measure_trading_intensity_and_volatility(hbt):
    arrival_depth = np.full(10_000_000, np.nan, np.float64)
    mid_price_chg = np.full(10_000_000, np.nan, np.float64)

    t = 0
    prev_mid_price_tick = np.nan
    mid_price_tick = np.nan

    # Checks every 100 milliseconds.
    while hbt.elapse(100_000):
        #--------------------------------------------------------
        # Records market order's arrival depth from the mid-price.
        if not np.isnan(mid_price_tick):
            depth = -np.inf
            if len(hbt.last_trades) != 0:
                print(hbt.last_trades)
            for trade in hbt.last_trades:
                side = trade[3]
                trade_price_tick = trade[4] / hbt.tick_size
                print(side, side==BUY, side==SELL, trade_price_tick, depth)
                if side == BUY:
                    depth = np.nanmax([trade_price_tick - mid_price_tick, depth])
                else:
                    depth = np.nanmax([mid_price_tick - trade_price_tick, depth])
            arrival_depth[t] = depth

        hbt.clear_last_trades()

        prev_mid_price_tick = mid_price_tick
        mid_price_tick = (hbt.best_bid_tick + hbt.best_ask_tick) / 2.0

        # Records the mid-price change for volatility calculation.
        mid_price_chg[t] = mid_price_tick - prev_mid_price_tick

        t += 1
        if t >= len(arrival_depth) or t >= len(mid_price_chg):
            raise Exception
    return arrival_depth[:t], mid_price_chg[:t]

What could be the possible cause?
Thank you.

DONE

TL;DR:
Feature request: progress bar for backtesting.

What I did:

1. Downloaded data using https://github.com/nkaz001/collect-binancefutures
2. Tried to run https://github.com/nkaz001/hftbacktest/blob/master/examples/Market%20Making%20with%20Alpha%20-%20Order%20Book%20Imbalance.ipynb on my dataset.
3. obi_mm(hbt, ...) became silent after the data was loaded.

So I don't understand if the backtest is even working. And what if it would require several days to backtest on my laptop?

It would be great to include a progress bar somewhere. tqdm package is a great tool to have pretty progressbars in raw python and in jupyter environments.

Code sample:

hbt = HftBacktest(
    # Add compressed data for other days if needed
    # for a more complete backtest
    [
        'data/eth_npzs/ethusdt_20230923.npz',
        'data/eth_npzs/ethusdt_20230924.npz',
        'data/eth_npzs/ethusdt_20230925.npz'
    ],
    tick_size=0.01,  
    lot_size=0.001,  
    maker_fee=-0.0001,  
    taker_fee=0.0005,  
    order_latency=FeedLatency(),  
    queue_model=SquareProbQueueModel(),
    asset_type=Linear,  
    snapshot='data/eth_npzs/ethusdt_20230922_eod.npz'
)

stat = Stat(hbt)
skew = 5
strategy(hbt, stat.recorder, skew)

stat.summary(capital=15_000)

plt.show()

The data is being fetched only when the strategy(hbt, stat.recorder, skew) is called.

However, I want to pre-load it so I can test different combinations of params afterwards.

Should I use hbt.__load_data after I create hbt?

Hi nkaz001,

Many thanks for this awesome project!

Quick question, my orderbook data is too large to fit in memory, therefore I need to chunk one single file to multiple chunked files.

The problem is that the result is different between reading a single file and reading multiple chunk files.

The way I test is that I first created a single file example btcusdt_20230405.npz using the file btcusdt_20230405.dat.gz by convert() as documented. And then I split that file into 7 subfiles. I then running these two experiments by changing hbt's input like this:

However, the backtest results are different:
Single File Result:

Chunked File Result:

One possible reason is that, I did some research and find it really weird that from the 1324th environment step (elapse step), the current_timestamp between chunked hbt and single hbt become different:

I double checked the underlying data, they are identical.

I have two questions:

1. Could you please think about the reason for this difference?
2. What is the best practice for loading huge orderbook dataset?

My email is [email protected]. I am more than happy to have a quick chat with you to help u replay this issue.

Many thanks!

suggestion for adding more exchanges

I have noticed that stat.equity() runs very slow. And it is used in your "sharpe", "sortino", "drawdown", "annualised_return" and "summary". When working in a sub-second intervals, this can run very slow because of many data points.

A simple njit equity function runs 100x faster:

@njit
def equity(positions, mids, balances, fees):
    result = List()
    for i in range(len(positions)):
        result.append(positions[i] * mids[i] + balances[i] - fees[i])
    return result

s = time.time()
equities = equity(stat.position, stat.mid, stat.balance, stat.fee)
e = time.time()

print(f'Fast computation took: {e - s}')

s = time.time()
stat.equity()
e = time.time()

print(f'Stat computation took: {e - s}')

Output:

Fast computation took: 0.11081385612487793
Stat computation took: 11.64174509048462

(2 million elements)

I have not looked deep into your code to find why exactly the equity function runs slow, I suppose it's because of pandas.

So i am trying to wok with data from Tardis. So far, all the binance-futures data from Tardis was correct and worked nicely. But now i am working with Bybit and got stuck. First i tought something is wrong with my strategy but more in depth research actually showed that it has something to do with the data itself. Maybe it is converted not correctly, or maybe it contained mistakes from the start at Tardis.

So here is what i get with simple setup:

@njit
def test(hbt):   
    tmp = np.empty((100_000_000, 3), np.float64)
    i = 0
    while hbt.elapse(60_000_000):
        hbt.clear_last_trades()
        tmp[i] = [hbt.current_timestamp, hbt.best_bid, hbt.best_ask]
        i += 1
        if i > 100:
            break
    return tmp[:i]

Output:

Load data/bybit/BCHUSDT/_2023-07-02.npz
bybit-BCHUSDT_Depth-5_Start-20230702-End-20230819
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125

When i switch to binance-futures, it spits out actual prices:

Load data/binance-futures/BCHUSDT/_2023-07-02.npz
binance-futures-BCHUSDT_Depth-5_Start-20230702-End-20230819

282.83500000000004 282.83500000000004
282.725 282.725
282.115 282.115
282.165 282.175
281.705 281.705
280.83500000000004 280.845
281.605 281.635
282.235 282.195
281.365 281.21500000000003
281.625 281.575

I tried examining npz file. It does contain information and each row is different, so no repeating values like those above.
I got the same output from this simple function both with and without specifying snapshot file.

For the sake of saving time, i also ran checks that we discussed in previous issue, with following results:

Rows where timestamp in column 2 is not bigger than timestamp in column 1:
740
Rows where timestamp in column 1 is not higher than in previous row:
0
Rows where timestamp in column 2 is not higher than in previous row:
0
Rows with zero timestamp:
0

I am confident that i am using correct tick and lot sizes for BCH at BYBIT (0.05, 0.01). Some other coins works nicely. For example, LPTUSD, ADAUSDT, ARBUSDT and others.

Here is a link to converted files: https://drive.google.com/drive/folders/141LdAXZwvY2mXjXuZmQrALL7Zaj1LUCe?usp=sharing

Hopefully, this issue can be resolved. Can't imagine doing the research without this tool.

Hi - I tried another experiment based on the simple_two_sided_quote example, where i adjusted the bid-offer spread of the orders I submitted to be 2 ticks wide, with idea to see some fills of my own orders (see the gist below). Then I used the very small data set attached (same as last issue I raised) and noted that at the time of the first trade event (buy @ 970), I have submitted limit orders buy 966 / sell 968. First query is that when I am debugging I don't see my orders in the market depth object on the exchange side (checked the time stamps, don't think this is due to latency). Also I don't see my order getting filled, which i think would be the expected behaviour?
Also note that I added depth clear events to my data set, as the data store I am dealing with just has a snapshot of the book to 10 levels at each time stamp, attached this data set below. I think this the right format? I cross checked with the Binance example.

thanks

Archive.zip

https://gist.github.com/griffiri/fd5600763f80a994febdd5eed99109ed

Usually, when testing multiple parameters of a trading strategy using multiprocessing, i create a list of variables, load np arrays and assign to those variables and then feed the list of those variables into backtester. But this only works for small time ranges and coins which do not have a lot of events per day.

When there is a lot of data or i am testing a big time range, i simply generate a list of file paths to data files and feed to the backtester, which are then loaded one by one. The problem is that each instance of a backtester loads the same file into memory which doesn't happen when data is taken from pre-loaded into memory np arrays. So sometimes tens of instances may load the same file into memory which hangs the hole process. As a workaround one has to limit the number of threads.

Is there a way to keep a file into memory once it was loaded by one of the instances? I was trying to come up with the solution, but still have no orking idea. In the perfect world, backtester should be aware that there are multiple threads of it is running and should make a loaded into memory file available to other threads. When all threads are finished working with the thread, it should un-load it from memory. This is what i was thinking, but it is hard for me to implement in Numba.

Hey, thanks for providing the Rust implementation!

Hey!

I am having problems converting data from Tardis for following assets/days:

Reading data/binance-futures/SOLUSDT/_trades_2024-01-09_SOLUSDT.csv.gz
Reading data/binance-futures/SOLUSDT/_incremental_book_L2_2024-01-09_SOLUSDT.csv.gz
Error occurred during conversion: index 1500 is out of bounds for axis 0 with size 1500

Reading data/binance-futures/TRBUSDT/_trades_2023-12-31_TRBUSDT.csv.gz
Reading data/binance-futures/TRBUSDT/_incremental_book_L2_2023-12-31_TRBUSDT.csv.gz
Error occurred during conversion: index 1500 is out of bounds for axis 0 with size 1500

I get the sam error on both dates:
Error occurred during conversion: index 1500 is out of bounds for axis 0 with size 1500

Which seem very strange to get the same kind of error on two different asset/date combinations.

P.S. Please, let me know if data upload somewhere on is necessary for debugging.

The function definition is:

def is_filled(self, order, hbt): return round(order.q[0] / hbt.lot_size) < 0

Why the left side must be smaller than 0? "<=" should work

Hi - to better understand the library I took a very small data sample (one minute of data, about 50 book updates and a few trades, in the attached archive) and tried to run it through the simple_two_quote function you have on the main page of the project. I've put a self contained script here https://gist.github.com/griffiri/12898ab4684e0d63375c8ea6dafabfa1, to run it you need to put the extracted contents of the archive into the same directory as the script.

When running the script in the first elapsed time interval orders are submitted as expected, then in the second elapsed time interval orders are submitted again. As the price in tick units has not changed the 'order id' has not changed. After this a duplicate order id error is thrown (error below). I thought the issue might be that the same order id is used, but the open orders are cancelled before the new orders are placed. Then I thought it might be because the new orders are placed before the exchange has had a chance to process the cancel, so I modified the script to use the wait=True option when cancelling the order. That does seem to process the cancel on the exchange side, but I don't see the cancel processed on the local side and I get the same error. I started looking into why but thought it might better to ask at this point, in case I am on the wrong track.

Traceback (most recent call last):
  File "/Users/richardgriffiths/projects/hftbacktest/examples/run_simple.py", line 115, in <module>
    run()
  File "/Users/richardgriffiths/projects/hftbacktest/examples/run_simple.py", line 109, in run
    simple_two_sided_quote(hbt, stat.recorder)
  File "/Users/richardgriffiths/projects/hftbacktest/hftbacktest/proc/local.py", line 113, in submit_order
    raise KeyError('Duplicate order_id')
KeyError: 'Duplicate order_id'

Archive.zip

Normally, the local process should receive the order acceptance information first and then the order execution information, as shown below:

exch process  2 0 1 1686067320067000
local process  2 1 0 1686067320081000
filled and del  2 1686067324745000.0
local process  2 3 0 1686067324751000
current_timestamp: 1686067380023000 , order_id: 2 , order_status: FILLED , order_req: NONE , order_price: 26097.2 , exec_price: 26097.2 , order_qty: 0.001 , exec_qty: 0.001 , leave_qty: 0.0

However, due to delays, it is possible to receive the FILLED information before the NEW information. In this case, the local process may mistakenly assume that a new order has been submitted, but the exchange process does not have this order because it has already been executed.

exch process  13 0 1 1686067980057500
filled and del  13 1686067980066000.0
local process  13 3 0 1686067980072000
local process  13 1 0 1686067980092000
current_timestamp: 1686068040023000 , order_id: 13 , order_status: NEW , order_req: NONE , order_price: 26159.800000000003 , exec_price: 0.0 , order_qty: 0.001 , exec_qty: 0.0 , leave_qty: 0.001

My suggestion is that when cleaning up the inactive orders (hbt.clear_inactive_orders()), it is necessary to consider deleting the orders that receive both the NEW and FILLED information for the same ID.

Hi @nkaz001, I have been playing with this backtesting framework for some time. I wanted to understand how can we check the trades that are executed when we are doing the backtest. What would be a standard process to debug the strategy results? Thank you

I would like to contribute. First I will try to understand your code.

Two questions:

• Why not just write C and build C bindings? Do you think that will improve the performance?
• I work on this library here: https://github.com/marsupialtail/quokka/blob/master/blog/orderedstreams.md. I do distributed stuff. I would be interested in supporting distributed execution of this library on cloud setup with data coming from S3. I actually tried to write something like you did in C/Python, but might as well just use your code now that I found it.

Hi,
I was trying to get the data prep working... when running the following step I get an error:

from hftbacktest.data.utils import create_last_snapshot

# Build 20230404 End of Day snapshot. It will be used for the initial snapshot for 20230405.
data = create_last_snapshot('btcusdt_20230404.npz', tick_size=0.01, lot_size=0.001)
np.savez('btcusdt_20230404_eod.npz', data=data)

# Build 20230405 End of Day snapshot.
# Due to the file size limitation, btcusdt_20230405.npz does not contain data for the entire day.
create_last_snapshot(
    'btcusdt_20230405.npz',
    tick_size=0.01,
    lot_size=0.001,
    initial_snapshot='btcusdt_20230404_eod.npz',
    output_snapshot_filename='btcusdt_20230405_eod'
)

results in:

TypeError: Failed in nopython mode pipeline (step: native lowering)
Method '__delitem__' is not supported.

date = '2023-08-08'

pair = 'ltc_usdt'
tick_size = tick_and_lot[pair]['tick_size']
lot_size = tick_and_lot[pair]['lot_size']

path = f'data/{pair.split("_")[0]}_tardis_npzs/{pair.replace("_", "")}_{date}_tardis.npz'

# Convert automatically cals validate data no need for us to do so
tardis.convert(["/".join(path.split("/")[:-1]) + "/tmp_ob.csv.gz","/".join(path.split("/")[:-1]) + "/tmp_trades.csv.gz"], output_filename=path)

Error:

    156 ss_ask = ss_ask[:ss_ask_rn]
    157 # Clear the ask market depth within the snapshot ask range.
    158 tmp[row_num] = [
    159     DEPTH_CLEAR_EVENT,
--> 160     ss_ask[0, 1],
    161     ss_ask[0, 2],
    162     -1,
    163     ss_ask[-1, 4],
    164     0
    165 ]
    166 row_num += 1
    167 # Add DEPTH_SNAPSHOT_EVENT for the ask snapshot

IndexError: index 0 is out of bounds for axis 0 with size 0

in file tardis.py

Here is a screenshot from the Debugger (when the ss_ask is empty):

Maybe a useful note will be that the timestamp and local_timestamp sometime may be a bit shifted wrt what tardis gives (at maximum - one microsecond (sometimes)).

I'm working on a project based on your research related to Grid Trading, but I added Support/Resistance to it to reduce the number of entries. Can I contact you via Telegram?

the data in the examples like
'data/ethusdt_20221003.npz',
'data/ethusdt_20221004.npz',
'data/ethusdt_20221005.npz',
'data/ethusdt_20221006.npz',
'data/ethusdt_20221007.npz'

Hi!

Feel a bit awkward for asking so many questions, but i am knee deep into using this beautiful piece of software and it provides an opportunity to learn HFT with lighnting speed for which i am sincerely greatful.

So i am using tardis as a source of data. And i notice, that when i try to run a backtest on a month worth of data, sometimes it gets stuck while loading data from some of the files. It could happen right on the first file or somewhere in the middle in the 'loading file' phase. So i just skip the file, generate new snapshot and it may help. Usually, i have to skip two days of data presumably because i need to make a new snapshot and a snapshot based on the data that can't go through loading phase would also not allow the backtest to work.

Providing sample files would be asking too much, so maybe you could point me into direction on where to look for? I mean, maybe there some corrupt values in the array that prevent backtester from processing further. I checked those files ащк nans or outliers and can't find anything that seems strange or somehow different from the data in files that load just nicely.

To exclude ill coded function as a cause of data not being able to load, i created this one and pass an instance of hbt to it:

@njit(cache=True)
def pazz(
    hbt
):
    interval = 1_000_000
    
    while hbt.elapse(interval):
        pass

And this is how i start it:

base = 'SUI'
start_date = 20230702
end_date = 20230731


hbt = HftBacktest(
    [
        'data/{}USDT/binance-futures_{}.npz'.format(base, '{}-{}-{}'.format(str(date)[:4], str(date)[4:6], str(date)[6:])) for date in range(start_date, end_date)
    ],
    tick_size= 0.0001, 
    lot_size= 1,
    maker_fee= -0.0001,
    taker_fee= 0.0005,
    order_latency=FeedLatency(),
    queue_model=SquareProbQueueModel(),
    asset_type=Linear,
    exchange_model=PartialFillExchange,
    snapshot='data/{}USDT/binance-futures_{}-{}-{}_SNAPSHOT.npz'.format(base, str(start_date-1)[:4], str(start_date-1)[4:6], str(start_date-1)[6:])
)


pazz(
    hbt
)

And i still get stuck at loading the first file. If i start from 25th it loads nicely to 31th.

Hi!

First of, thank you for sharing this amazing piece of software.

I am fairly new to HFT and i am probably doing something wrong. I tried to simulate a Grid Trading strategy with a skew on a few coins. Two of them are ARBUSDT perpetual and BTCUSDT perpetual. When i backtest the strategy on BTCUSDT, i get this graph:

This is one day worth of data, strategy is used as it is presented in the examples folder. Here is how it is executed:

But when i try to simulate the same strategy on ARBUSDT on one weeks worth of data, here is the graph that i get:

It provides statistics in a text form, but doesn't provide cumulative returns neither with nor without fees.

Here are settings that i use to start simulation:

Data for both ARBUSDT and BTCUSDT are taken from Tardis via CSV downloads. After that i convert it using tardis.convert(). When creating a snapshot i use data from previous day compared to the day backstart starts with:

create_last_snapshot(f'data/ARBUSDT/{exchange}_2023-07-18.npz', 
                                tick_size=0.1, 
                                lot_size=0.000001,
                                output_snapshot_filename = f'data/ARBUSDT/{exchange}_2023-07-18_SNAPSHOT.npz')

What am i doing wrong?

In High-Frequency Grid Trading.ipynb, shows:

Just wonder why cumulative returns of strategy incl fee is higher than strategy excl fee?
I think that strategy incl fee is strategy with fee, which should has less cumulative returns.

I came across this amazing project while looking for a HFT-backtesting framework. I can understand this involves a Single Asset Market-Making Model. Is there any way to incorporate some underlying asset order book and give a logic for calculating fair for derivatives?

It would be interesting to see this feature because currently, I'm just creating a time series of underlying assets on which the derivative market depends. Then getting the last timestamp from the stat.timestamp list to get underlying value to determine what jump to apply on the Derivative fair and then put bid-ask prices that are to be backtested for market-making derivative.

I think this feature should be easy to accommodate in this framework. This can make it a pretty interesting proposition backtesting market-making strategies for ETFs, Futures, or Options as well.

I followed the example in the document caculated the lambda of Guéant–Lehalle–Fernandez-Tapia Market Making Model. I found the market order’s arrivals does not match the Poisson distribution. How can I better calibrate the lambda .

Hi - thanks for sharing this repo. I noticed that this link is broken https://github.com/nkaz001/hftbacktest/tree/master/datautils, I suppose the functionality is similar to what is in https://github.com/nkaz001/collect-binancefutures/blob/master/convert/convert.py? Anyway would be great to fix up the link (it's on the main page and the data page)
thanks
Richard

Hi @nkaz001
How can I use my csv tick_data from nasdaq indices into your jupyter-notebook for using strategy?
Thanks :)

Hi - I noticed that the even when using a small sample data file it is quite slow to run (say a few mins), because the time spent compiling the jitted code dominates. This makes it a bit frustrating iterating through different trials, did you find a way to avoid this cost? I was wondering if it is possible to compile ahead or drop back to pure python.
thanks

Hi there! Just spotted this repo recently and been trying to set it up and play around with few options of own.
But - seems like git-lfs used to track *.pkl files are causing some trouble in cloning the repo and throwing errors.

Would it be possible to update the references or another file option to make this repo easily pull-able please?
Appreciate the work on this framework!

Stacktrace:

(.env.3.9) ➜  hftbacktest git:(master) ✗ git reset --hard
Updating files: 100% (22/22), done.
Downloading example/usdm/bnbusdt_20221116.snapshot.pkl (19 KB)
Error downloading object: example/usdm/bnbusdt_20221116.snapshot.pkl (af67be7): Smudge error: Error downloading example/usdm/bnbusdt_20221116.snapshot.pkl (af67be7664c09d002afe64f8b2a956f87c0355571ae8a0bbcf818d2e9f3cab04): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Errors logged to '/Code/hftbacktest/.git/lfs/logs/20230222T150946.286206.log'.
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: example/usdm/bnbusdt_20221116.snapshot.pkl: smudge filter lfs failed

I checked out the library you have linked (https://github.com/nkaz001/collect-binancefutures), but it seems that it is gathering data live.

Regarding Tardis, they allow to get the data for the first day of each month, which is not enough for a meaningful backtest.

I see in your documentation you have data for multiple days, how did you get it?

Regards

Thanks for this great project.

How would you move from backtesting to live without having to re-implement the strategy logic and introduce bugs in the process?

For example, ideally def market_making_algo(hbt) could be used in sample-trading-bot.

Hi - I am going to try out a market making strategy where one of the inputs will be the price on another exchange and would like to try to run a back test. I was wondering whether this should be done using the 'user' event type in the data feed? Or simply import a file with the data when writing the algo. I couldn't see an example of this sort of thing in the repo. thanks

I have orderbook data that looks like this:

timestamp, price, side, volume
2023-02-10 23:59:49.789,21637.48,ask,0.67350
2023-02-10 23:59:49.789,21630.66,ask,0.03474
2023-02-10 23:59:49.789,21619.44,bid,0.00000

(0 volume here means cancel order/match order, doesn't matter)

I want to simulate trades on secondly basis. In each second I can potentially place an order (a market order or a limit order) so that they can be processed by your package. Is that possible?

Regards

Hi - minor comment re the docs, I believe the timestamp unit is expected to be in microseconds but I don't see that mentioned in the documentation. It took a few minutes to figure out i had the wrong units for timestamp when i was trying an example, probably worth mentioning.
thanks
Richard

Hi,
Thanks for providing this great project. Currently, I'm trying the "Making Multiple Markets" example. Could you kindly share the format or the example of 'assets.json' file to speed up my testing?

Thanks a lot.

Hi nkaz, please correct me if i'm wrong.

In current implementation, when clear_depth event hits, it clears "Clear the bid/ask market depth within the snapshot bid range."

However, the clear up to price is assigned as ask's best price and bid's worst price in the snapshot:

And in marketdepth implementation, it clears from current best price to clear up to price:

In this case, it clear's current best ask to snap best ask , while clears current best bid to snap worst bid. which leaves an inconsistency.

I have two questions:

1. May I ask why is clear event needed?
2. Shouldn't it always be current best to snap worst for both ask and bid?

many thanks!

After each hbt.elapse(micros), all hanging orders are marked as inactive and can be removed using hbt.clear_inactive_orders(), right?

I just want to make sure that the elapsing marks orders as inactive because it is intuitive, but I cannot find the part of code that does it.