want to find the best way to convert websocket data to candle data (any time frame)pi

Automate, API & MCP
1

I’m using following python code to convert ticks data to construct candle data.

constructing a dataframe with websocket data .

def custom_message(msgs): 
    for msg in msgs:
        ltp = msg['ltp']
        time_stamp = datetime.fromtimestamp(msg['timestamp'])
        instrument = msg['symbol']
        exchange = 'NSE'
        global df_data
        tmp_df = pd.DataFrame([[instrument, ltp, ltp, ltp, ltp, time_stamp, exchange]], columns=['symbol', 'high', 'open', 'low', 'close', 'timestamp', 'exchange'])
        df_data = pd.concat([df_data, tmp_df], a

conversion = {'open' : 'first', 'high' : 'max', 'low' : 'min', 'close' : 'last'}
copydf = df_data.copy(deep=True).drop_duplicates()
copydf['timestamp'] = copydf['timestamp'].apply(pd.to_datetime)
copydf.set_index('timestamp', inplace=True)
sample = copydf.resample('3Min').agg(conversion).dropna()

Is there any better way to do that?

2

I have posted outline of my code here

For OHLC Data Capture -Assistance required for transitioning from Fyers-V2 to Fyers-V3

The dataframe holds data only for current bar in the timeframe I prefer.

Then at suitable interval like every sec i push the bar into charting tool or you can update your DB.

If you intend to store all ticks in DF, then you have to be more innovative.

3

Hi @nag.046bes5o ,

Websockets offer real-time candle values for the current day. To obtain values for other resolutions, you’ll need to develop the logic for that.

4

Hi @nag.046bes5o

Here is my version. Hope it will help

def custom_message(msgs): 
    for msg in msgs:
        if  msg.get('ltp'):
            tmp_df = pd.DataFrame.from_dict(msg, orient="index").T
            global df_data
            df_data = pd.concat([df_data, tmp_df])

copydf = df_data.copy(deep=True).drop_duplicates(subset = ["last_traded_time", "ltp"])
copydf["timestamp"] = pd.to_datetime(copydf["last_traded_time"], unit= "s")
copydf.set_index("timestamp", inplace=True)
copydf['ltp'] = copydf['ltp'].astype(float)
sample = copydf.resample("3Min")['ltp'].ohlc()
5

Thanks Chirag - let me go through the code.

6

real time data or candle values?

7

our code is more or less same :+1:

8

Nice @nag.046bes5o

You are getting candle now ?

9

yes, but want to know the best way to do it

10

Hi It is nice,
If I got some new way , I will share

11

Hi @subhajit_bhar - If I have to add the timestamp, how to add it (I’m referring your code below)?

tmp_df = pd.DataFrame.from_dict(msg, orient=“index”).T

12

Hi @nag.046bes5o ,

As per your code, msg has a key, viz, ‘timestamp’. If so, then, tmp_df will have a column for ‘timestamp’.

13

yeah…I was working on some other code where there was no time component, but I have figured it out. Thanks for spotting.

14

Hello Nag,

If you can share the code to convert the websocket data into ohlc