Use Compact json formatting for storing document metadata
ClosedPublic

Authored by bruns on Dec 2 2018, 3:40 PM.

Details

Summary

Document metadata is stored json encoded in the documentdatadb for fast
retrieval. The default pretty-printed format adds useless whitespace:

$> mdb_dump -p -s documentdatadb -n ~/.local/share/baloo/index
{\0a "26": 3648,\0a "27": 5472,\0a "30": "Canon",\0a

"31": "Canon EOS 70D",\0a    "32": "2017-11-12T17:42:35.000Z",\0a
"33": 1,\0a    "34": 9,\0a
"35": [\0a        3648,\0a        5472\0a    ],\0a
"37": "2017-11-11T16:54:57.000Z",\0a    "38": 50,\0a
"40": 0.01666666753590107,\0a    "41": 3.200000047683716,\0a
"42": 3.375,\0a    "43": 0,\0a    "44": 0,\0a    "45": 5,\0a
"46": 800\0a}\0a

Using the Compact encoding reduces the storage size significantly:
{"26":3648,"27":5472,"30":"Canon","31":"Canon EOS 70D",
"32":"2017-11-12T17:42:35.000Z","33":1,"34":9,"35":[3648,5472],
"37":"2017-11-11T16:54:57.000Z","38":50,"40":0.01666666753590107,
"41":3.200000047683716,"42":3.375,"43":0,"44":0,"45":5,"46":800}

Diff Detail

Repository
R293 Baloo
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
bruns created this revision.Dec 2 2018, 3:40 PM
Restricted Application added projects: Frameworks, Baloo. · View Herald TranscriptDec 2 2018, 3:40 PM
Restricted Application added a subscriber: kde-frameworks-devel. · View Herald Transcript
bruns requested review of this revision.Dec 2 2018, 3:40 PM
astippich accepted this revision.Dec 2 2018, 6:24 PM
This revision is now accepted and ready to land.Dec 2 2018, 6:24 PM

This is backwards-compatible with the existing formal for retrieval, right?

bruns added a comment.Dec 2 2018, 7:32 PM

This is backwards-compatible with the existing formal for retrieval, right?

From the json standpoint, both are equivalent.

ngraham accepted this revision.Dec 2 2018, 7:35 PM

Might this make the DB take up less space on disk?

bruns added a comment.Dec 2 2018, 7:46 PM

Might this make the DB take up less space on disk?

Yes, but not in a significant way - for me, the docData is less than 1% of the DB, this saves 10..50% of that.

This revision was automatically updated to reflect the committed changes.