# Decode SQLite database blobs: how to start?

Updated: see bottom of this question.

Not sure if I am at the right channel here but I have good hopes someone here might be able to help me.

I am trying to process and analyse data from a Footscan system that is exported from the Footscan 9 Gait Essentials software to a .rsdb database. This database is a standard SQLite database with a different file extension.

I have no problems accessing the data via Python but the most interesting data (the raw data from the sensor) is stored as a 'blob' (see sqlite documentation for more details).

Exporting this data from SQLitePro to .csv shows that the data in the blob looks like this (although much longer):

<9c4e79cb cbd09de5 c380fc78 a74e819f ... ab2172c7 d9cf311a 357aebac>


My question is: what would be my first steps to try to discover how this data is encoded? And does anyone maybe recognize this type of encoding?

Extra hints: The data represents all forces of a force plate with 64x64 force sensors, sampled at 300Hz, so it's most probably a multidimensional array.

Update

I am making some progress. As commented below, the values are most probably hexadecimal values, but I'm not sure if the values are base 16: most values seem to be >10^9...

Furthermore: I found a table in the database called 'Contact' which linkes to entries in the blob table which contain the most hexadecimal values (>40000). My guess is that these are the entries which are recognized as actual steps. I extracted the data from one of these steps here (Contact id 2; blob id 24). This blob contains 50792 hexadecimal values.

This is the entry for this step in 'Contacts':

id: {35d9a73e-608d-4ba3-9823-de0540f11c93}
timestamp: 2
deleted: NULL
FrameWidth: 64
Frames: 24
FrameOffset: 12204
FrameCount: 214
FrameHeight: 64
FrameStoreWithContacts_Contacs: {5d10546e-5ef3-4469-b898-d7b9bd58458f}
OriginalId: {32ae0f73-8022-4151-92e7-510c0e86d09d}
orphan: false


Full export of the blob table can be downloaded here. The full .rsdb can be found here (you might have to rename to .sqlite). The tables 'Foot' and 'Region' in the full database might contain useful data too, but I'm not yet sure how...

• Do you mean the encoding beyond the hexadecimal system of encoding bytes, or have you never seen hexadecimal before? – Spacedman Feb 13 '16 at 16:15
• I have seen hexadecimal before but I am not sure if I understand your question completely. Can you elaborate? – Aart Goossens Feb 14 '16 at 11:15
• Convert the data to binary. If the data's not private, post it and I can show you what I mean. The first few bytes you provided don't suggest a file type, but emacs thinks it might be Japanese/JIS text. – user4710 Feb 14 '16 at 15:49
• Its not clear from your Q whether you understand that "9c4e79cb" means anything to you (ie you understand its hexadecimal). So there's (at least) two "encodings" going on - first the representation of the data as hex, and then however the force plate measurements are arranged into that data. It could just be 64x64xN bytes. There's not really enough context in the Q to figure it out... – Spacedman Feb 14 '16 at 23:22
• I think its a bit tricky ... blobs can be of any structure inside. It helps that you expect a certain array of data (64x64) sampled at 300Hz, though how long is the data stored for? If you know how many seconds it is stored, and you know the blob size, you can then estimate the individual point resolution. – Marcus D Feb 15 '16 at 13:22