GitHub - crazy-max/firefox-history-merger: ? Merge Firefox history from multiple...

Abandoned project

This project is not maintained anymore and is abandoned. Feel free to fork and make your own changes if needed.

About

firefox-history-merger is a CLI application written in Go to merge history of Firefox and repair missing favicons with ease.

Screenshot of firefox-history-merger

Features

• Merge history (moz_places) from places.sqlite file
• moz_historyvisits are also merged
• Frecency is recalculated during the merge
• Repair favicons
• Display info about places.sqlite
• Optimize database into a minimal amount of disk space

Requirements

Schema version that can be used is v39 (Firefox 57) to v53 (Firefox 81). Check if your places.sqlite is compatible with info command.

If your database is not compatible, you can copy your places.sqlite in a compatible Firefox profile folder, then launch/close Firefox and your places.sqlite should now be compatible.

Download

You can download the application matching your platform on the releases page.

Usage

First close Firefox and copy places.sqlite and favicons.sqlite files from your Firefox profile folder somewhere.

In the following examples, this folder structure will be used:

favicons.sqlite
firefox-history-merger.exe
places.sqlite
places_to_merge.sqlite

You probably want to have more info about your places.sqlite database you have before merging history:

$ firefox-history-merger info places.sqlite
Sun, 23 Jun 2019 03:43:47 CEST INF Schema version:         v52 (Firefox >= 65)
Sun, 23 Jun 2019 03:43:48 CEST INF Compatible:             true
Sun, 23 Jun 2019 03:43:48 CEST INF Places entries:         1934
Sun, 23 Jun 2019 03:43:48 CEST INF Historyvisits entries:  1678
Sun, 23 Jun 2019 03:43:48 CEST INF Last used on:           2019-06-21 02:31:46

Merge

Now execute this command to merge places_to_merge.sqlite with your working places.sqlite:

$ firefox-history-merger merge places.sqlite places_to_merge.sqlite
Sun, 23 Jun 2019 04:33:58 CEST INF 5199 places will be merged
Sun, 23 Jun 2019 04:33:58 CEST INF Merging 1000 places (1/6)...
Sun, 23 Jun 2019 04:33:59 CEST INF Merging 1000 places (2/6)...
Sun, 23 Jun 2019 04:34:01 CEST INF Merging 1000 places (3/6)...
Sun, 23 Jun 2019 04:34:03 CEST INF Merging 1000 places (4/6)...
Sun, 23 Jun 2019 04:34:05 CEST INF Merging 1000 places (5/6)...
Sun, 23 Jun 2019 04:34:07 CEST INF Merging 199 places (6/6)...
Sun, 23 Jun 2019 04:34:09 CEST INF Optimizing database...
Sun, 23 Jun 2019 04:34:09 CEST INF Finished created=4478 errors=0 total=5199 updated=721

Repair favicons

If you want you can also repair missing favicons. It can take a long time depending on the number of entries in your history, since the favicon will be recovered for each entry if there is no attached favicon.

$ firefox-history-merger repair-favicons places.sqlite favicons.sqlite
Sun, 23 Jun 2019 04:29:02 CEST INF Checking 1948 places...
Sun, 23 Jun 2019 04:29:02 CEST INF Favicon repaired favicon_id=18575 left=1938 places_id=33864 url=https://github.com/
Sun, 23 Jun 2019 04:29:05 CEST ERR Cannot get favicon error="no favicon found" favicon_id=0 left=1845 places_id=200497 url=http://www.videojs.com/
...
Sun, 23 Jun 2019 04:32:54 CEST ERR Cannot get favicon error="no favicon found" favicon_id=0 left=1 places_id=1955773 url=https://docs.docker.com/v17.09/engine/admin/logging/view_container_logs/
Sun, 23 Jun 2019 04:32:54 CEST INF Finished errors=618 linked=873 repaired=402 skipped=23 total=1948 valid=32

About Firefox places

Places expiration limit

As stipulated in this StackExchange thread, the relevant function is _loadPrefs() in nsPlacesExpiration.js. It tries to determine the limit by first checking PREF_MAX_URIS (which is "max_pages"). If PREF_MAX_URIS isn't set or is a negative value, it then attempts to calculate the 'optimal database size' based on available memory and disk space.

So if you want to retain all history ('infinite' history), you could set places.history.expiration.max_pages to 2147483647 (the highest 32-bit signed integer) in about:config.

Once you create it and set it, check the value of places.history.expiration.transient_current_max_pages. It should automatically adjust itself to match your max_pages setting.

Databases schema

Database schema version is stored in user_version pragma statement. Linked Firefox version to database schema can be found in Database.cpp.

Edit this diagram in your browser.

Contributing

Want to contribute? Awesome! The most basic way to show your support is to star the project, or to raise issues. You can also support this project by becoming a sponsor on GitHub or by making a Paypal donation to ensure this journey continues indefinitely!

Thanks again for your support, it is much appreciated!

License

MIT. See LICENSE for more details.
Icon credit to Zlatko Najdenovski (firefox icon) and BomSymbols (clock icon).