kemitix/paperoni
Archived
1
0
Fork 0
Code Issues 4 Pull requests 3 Projects Releases Packages Wiki Activity
I've switched to using https://readeck.org/en/, even though it isn't a CLI, it does produce very nice epubs
This repository has been archived on 2024-11-22. You can view files and clone it, but cannot push or open issues or pull requests.
52 commits 5 branches 0 tags 1.7 MiB
Rust 99.8%
HTML 0.2%
Find a file
Kenneth Gitere 21c3ffd922 Refactor fetch_url 
This adds:
- More validation of responses to ensure the HTML response is valid.
- Better handling of redirecting URLs which allows for fetching of
  links proxied to Medium.
2021-01-24 17:52:31 +03:00
src Refactor fetch_url 2021-01-24 17:52:31 +03:00
test_html Add moz_readability initial code and accompanying unit tests 2020-08-31 19:30:09 +03:00
.gitignore Merge branch 'master' into dev 2020-05-16 10:35:47 +03:00
Cargo.lock Bug fixes 2020-12-24 14:03:36 +03:00
Cargo.toml Bug fixes 2020-12-24 14:03:36 +03:00
LICENSE Initial commit 2020-04-30 08:06:07 +03:00
paperoni-dark.png Add README 2020-10-22 16:03:57 +03:00
README.md Update README and version 2020-11-24 18:39:51 +03:00

README.md

Salami not included

Paperoni is a web article downloader written in Rust. The downloaded articles are then exported as EPUB files.

This project is in an alpha release so it is pretty unstable.

Usage

paperoni https://en.wikipedia.org/wiki/Pepperoni

Paperoni also supports passing multiple links as arguments. If you are on a Unix-like OS, you can simply do something like this:

cat links.txt | xargs paperoni

How it works

The URL passed to Paperoni is fetched and the returned HTML response is passed to the extractor. This extractor retrieves a possible article using a port of the Mozilla Readability algorithm. This article is then saved in an EPUB.

The port of the algorithm is still unstable as well so it is not fully compatible with all the websites that can be extracted using Readability.

How it (currently) doesn't work

This program is still in alpha so a number of things currently break:

  • Certain links with redirects can't be extracted. Such links include urls that are proxying Medium.
  • Websites that only run with JavaScript cannot be extracted.
  • Website articles that cannot be extracted by Readability cannot be extracted by Paperoni either.

Running locally

Precompiled binaries

Check the releases page for precompiled binaries. Currently there are only builds for Debian and Arch.

Building from source

This project uses async/.await so it should be compiled using a minimum Rust version of 1.33. Preferrably use the latest version of Rust.

git clone https://github.com/hipstermojo/paperoni.git
cd paperoni
## You can build and install paperoni locally
cargo install --path .
## or use it from within the project
cargo run -- # pass your url here