GitHub - croqaz/clean-mark: Convert an article into a clean Markdown text
source link: https://github.com/croqaz/clean-mark
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
➹ Clean-mark
Convert a blog article into a clean Markdown text file.
Example
For example, this article:
Is converted into this text file:
Usage
$ clean-mark "http://some-website.com/fancy-article"
The article will be automatically named using the URL path name. In the case, above, the name will be fancy-article.md
.
The file type can be specified:
$ clean-mark "http://some-website.com/fancy-article" -t html
The available types are: HTML, TEXT and Markdown.
The output file and path can be also specified:
$ clean-mark "http://some-website.com/fancy-article" -o /tmp/article
In that case the output will be /tmp/article.md
. The extension is added automatically.
Installation
Simply install with npm:
$ npm install clean-mark --global
- to save interesting articles offline, in a highly readable text format
- it's easy to read on a tablet, or a Kindle (as it is, or exported to PDF)
- Markdown is easy to export into different formats
- for offline text analysis of multiple articles, using machine learning / AI
Implementation steps:
- Downloads the content of a web page
- Meta-scrape page details (title, author, date, etc)
- Sanitizes the ugly HTML
- Minifies the disinfected HTML
- Converts the result into clean Markdown text
This project depends on the A-Extractor project, a database of expressions used for extracting content from blogs and articles.
Vision
The goals of the project are are:
- Good text extraction
- More useless text is preferred, instead of cutting from the actual article by mistake
- Extracting media (images, videos, audio) is not that important
- Extraction speed is not that important
Contributing
Clean-mark was tested on all major news sites. On some websites, the text, or links are cut from the article. In this case, you have to manually edit the resulted text,
please raise an issue on A-Extractor with the link that doesn't work and we'll add it in the database, so that next time, the text will be extracted correctly.
Also, see how to contribute.
Other readability tools
Web archiving tools
- https://github.com/Y2Z/monolith :: Rust CLI app
- https://github.com/go-shiori/obelisk :: Go CLI app, inspired by Monolith
- https://github.com/wabarc/cairn :: TypeScript implementation of Obelisk
- https://github.com/gildas-lormeau/SingleFile :: Web Extension for Firefox/Chrome/Edge and CLI tool to save a web page in a single HTML file
A massive list of Awesome Web Archiving tools: https://github.com/iipc/awesome-web-archiving
License
MIT © Cristi Constantin.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK