From 6aef1631e32210e6c7021333300cbc3833923b4b Mon Sep 17 00:00:00 2001 From: Kenneth Gitere Date: Thu, 22 Oct 2020 16:03:57 +0300 Subject: [PATCH] Add README --- README.md | 53 ++++++++++++++++++++++++++++++++++++++++++++++ paperoni-dark.png | Bin 0 -> 8123 bytes 2 files changed, 53 insertions(+) create mode 100644 README.md create mode 100644 paperoni-dark.png diff --git a/README.md b/README.md new file mode 100644 index 0000000..81aeff4 --- /dev/null +++ b/README.md @@ -0,0 +1,53 @@ + + +

Salami not included

+ +Paperoni is a web article downloader written in Rust. The downloaded articles are then exported as EPUB files. + +> This project is in an alpha release so it is pretty unstable. + +## Usage + +```sh +paperoni https://en.wikipedia.org/wiki/Pepperoni +``` + +Paperoni also supports passing multiple links as arguments. If you are on a Unix-like OS, you can simply do something like this: + +```sh +cat links.txt | xargs paperoni +``` + +## How it works + +The URL passed to Paperoni is fetched and the returned HTML response is passed to the extractor. +This extractor retrieves a possible article using a port of the [Mozilla Readability algorithm](https://github.com/mozilla/readability). This article is then saved in an EPUB. + +> The port of the algorithm is still unstable as well so it is not fully compatible with all the websites that can be extracted using Readability. + +## How it (currently) doesn't work + +This program is still in alpha so a number of things currently break: + +- Links with redirects will crash the program as it has no redirect logic. +- Websites that only run with JavaScript cannot be extracted. +- Website articles that cannot be extracted by Readability cannot be extracted by Paperoni either. + +## Running locally + +### Precompiled binaries + +Check the [releases](https://github.com/hipstermojo/paperoni/releases) page for precompiled binaries. Currently there are only builds for Debian and Arch. + +### Building from source + +This project uses `async/.await` so it should be compiled using a minimum Rust version of 1.33. Preferrably use the latest version of Rust. + +```sh +git clone https://github.com/hipstermojo/paperoni.git +cd paperoni +## You can build and install paperoni locally +cargo install --path . +## or use it from within the project +cargo run -- # pass your url here +``` diff --git a/paperoni-dark.png b/paperoni-dark.png new file mode 100644 index 0000000000000000000000000000000000000000..8339a4809ea8648cfde98effbdd5ee93c71deaa6 GIT binary patch literal 8123 zcmeHsRaBJGyElq}C?QCvgs3z~4@I3Y|F8E)! zq`eCs@a^Swov^S-GjINGRi#To!Hd++GCIy0c4p3Q#*U^~ZflaQ)Dd9cJmUc(f>XEkLpmt=ZY;YEq!K$ zRL_cUf#};<&%Ee)q=6cb%gFgHID z7w_!t$2Xv(H@16x`Oz{@HXzK@^rWPkOp=mp{-v*kx}st`T3H$^^Y+R0t@VuZa-Q3_ zZxcw^si~3NyLT@-IXUE;`ABv}#oHaH*8LwG_HJ&85S4fD$hFG!@grn&UTPEPs~0_B zV6csD*w$84`$1ci&^K<}crFXl#7HzubS%#X?6eilIA(MNS&=I*35( zcpF1FL-N+ZAj04OwtkZ<;rjZzysB#ACBw7sfl-nrXmxvi0@vv9EbPc#n}faLu!peA zpe<5TNs(-AmvMG>)_oz65+32OiQ?+&>0uuqmFx+p4siWD)lah&C3*EyT|L&qW@Rwh zV!SY>bZHgknmhrC^<*?RDf1VcZ1Vv^p`K6(#72tyFwm% zCa-S&{Jh0U&rz;uEiQ&XekQcL=Xe?&(=u%2r*I1;%$KkltG=_OlRW5gq(YmZZv zc!mf++?-Upa!g4~#G?$HE&2V;+~m)X_|R~f-1~MjEMZydML%m0+Jj#qw7pS`z3XEI zriK%vb@&h2sG4vBoE#ja?Cp7P?u*N~)R_4Az`j0t z5fKq8xa}O<8~d3_%5$%yal}j953O%f-yfSqE&O|^+CKMNx7=w$Mp-$$QtwgKU4nji zC4=9E11l@*{AWhrmnB-ra7<(&rJ*;;n-l>)S6bPzYDl=y_KXawW3p?fr8Qh01gaTc2x_ z^bZJ#z~ILGHj!rwqa-8yw7d-e_jh^{jU1m}TkEqA7MHR^qG zuu@Rf!UMqq)r_EwjEq@V&y%b3oxQDZBdB%3(b3W1(9mB>k~Mn^pBdjDZT%Kx05>fL zt+jrJu$GgPt8ZxNn4YG+b?eq$BBG$Wy1L;>3jrDeh8I+L#0;ee-k<(`7Yt>$KC-+OxtV7F9ojMD3b*;3b9ZXZu4#|8ZSW^Ri2M=6mzBxpepfyy)= z`5HUWu`-eo^;jvPFGX(F!=Hqtl!GkD8jA zkHp1k6%-WQH<5KeD^27E(?m7Qv25pBA+x^ce?5mN$BHxp?)G8vR(Uvi|E?*SMJXa` zpnxIdQJtNgqoumfAww$g--8DQ+%cch(&%nx?&;IJb+(vTZTC|uBBvA3a|Fn3Zi20aG}*}$cnr&K7Do;?w7JFWSP{ig zO|EOR4rM2yq{NGmb=uj%!NCCx`M{?vBZINg^>4AfDxrdk3Q=raoD2d6gMlCIT04{Y zRm0i?&HG*egUO03s4{A~ZimZyPDQY>vB~b|FSWP7^sRc8`zQ!r+r|TY1o(fw2l0G^C-53ADcxQK^6X=)s5xvKM zr$29PS&vSxP00?Y{aIcPmMH7;f)o6mstEwi2d%LR1Z;OQ%UBdO_9nfIi`<2N{-8Gb zD)xDuj~9CVS!tQ>H{3*S15za92yNoNo|pKKpC={{XyV&%_4PT=>eJKHw<=xe?=ZV` z*EcrOqzc)PBG<*DyV4X|9h!Q)78t%IRmV8q1xgrcKkWA3qe+y;$0=ug)#T+wJ^W`Efw&?}1GNTPuSwH8rj3 zO@K0}sjK?~PEu%1@M6189hW;g6g^7%GO5^$@1d@D5-~Xzdq;z>Oy+(|M2pR|P@1Td zY&ELY$E*43Y|}F}wNIUwHLlwU=)su{E*>6L6dLrH-0E<;3_{#>>!pH1$TDIiLo#Zq zH!2JjL`2U{y)_iCc{lEyA%-7t>0<;nHMJmvQ&nvb?{i6&Q%hlC_ous-U#NhnJgDFB zI9So0Y2A=kyrFnuqjs$R6g2U1W8vS~J{OT|vegkk7Sf0IyC7u}%It()dG&+yTz zI_hQg8~4(G^5n^a>)A$$XVl;I^$@85+z`|em>nQ8$306yYp`%|C{+u&Zt0-W=<$-?p!UvA z?7h9c*z;FU9zPaDbB)n7USF(Aia4zhoE`n;I87UoI9LAH^COaosN!An7`0cU%V?!) zH7P$o|8D=2j9^qL5?N{eo4;jP6<#)YT)+`?hl7JdT1LirG+QQi1OQDBP!k&vIUOCH z^kdnC>`{vf%Q3dWRN->jA@JKHFAfM=&$pY-sx5zc{#*N4-gX6DP5BYeq0KI2SfG+# zVT%##?irIoYMGq9#W)vR9_07@pLh?P8BV`)B~^{VQzcXWL9&1}e-fyYM@sTI~A_e9BdWb&2jbapbk zj}5k4+suqUr(Uh{ezAkHGQq{g1)PhCrAh!mMAZ$P!CE{g&JVS;wCtUnUIS$?3hwpJ zojaKt1uDee7(gL#q!Yb>#RD?lL`} zJA2djRu6-Owa|N~FzK9=A#dllWZCgN|M|$N0Q?fD&ND#uS7-uUh_8W@V zHR;39>P{lMUsVC-1&R=;lnu`)14F078?`?LT^bYD$(+XWUV7?sGW>kkH@aKb9!-|O+8dp(fs>$sShj|CaNG=cmmA|cxu zDi+!B2gtnK+<9RCkjH3%c+oHXf0Md4U~S)7*8OH?+#5*`B>Eiy-n@>>^0uO8a9Uwo zX&@ggkWQ<^xO(c+a3ZSM5UjKp9;80!N8Nu05=|qXgSUz(Slc{aW#`8tBPVB)kbqUA zzL~4`chU{YSXw^oiKIt-;aFjPgVbUX5~2Vm7!9l*5~jiFSRX9MINH!eOoHi%R;DP*&AkIw-vu;{sUf7b1auM=uh9n! z8=&Uv9hQQ&L~pF|Kq5~Lpd&%6tgNiN#Kdx@rg>W)%H6>I?#^9bQR3s{d$w_01Bp!~ zVxI|Hq=D#wG&VLC@N?-Jow2cT zwq!KcsS?-rFAD*5y#ou7vW7V*AhQXm`K9-%=7S#b@DR6M?B6G%7yH!JB@KimO;aAO z-TJRQGzd?yGbXcMXXkkerAT4z)rj50ub>HpOWi(x^G zGkC5sH6IL%Ckff4beuC=$y3Gl!*5jjbgfNQPv6LhoUt(zw_$7PBu6QjQYv6BEZ*dd zW&(QJ-QQP-a~pU~s3h@rWHO0@`^u4xh!vCtt)r-@7#$zqJ$4Yj_A5_Gr{11P;$%L6 zMi7BOfcPXEa89T<;Rai~1gE^%T{{yLHrhUW0@}xaxnOOcX#(K+)~a)Q%g%I={T6ddZPH< zT|z<`u;*Zn9?K+y>EhZ)9jsWHD61HtM7~nkW9w>ph&o18|ABgXib-5wIE`}j)YjGp zM@A;hM0^geZ^DUTW>ZPC*rLFiIqf+9rNl$sL}-->sF&AaQCd@BtLFhZdEhptiGzPq z6$d9Y++^d3_H--;U`tkEp?tNF_lZ4#pOJQY5`oYUev%>jdh%A?Q>2_G_1Qg1)s){Z*a=hf^;ifbO85wy*GC$Z$0 znf~u{bBsVtG=CIlwxR-P0Oya-qd2~Q|Ne8=`{N(tzkHdmnS{la^1zy@(#2e21?4PV z1Q(qg;St6gv5rm=RhOx6fzBGPS?1&8tEjBV`5s~wFh?%U@9yQ*3A|19(X?IL!i})m zyZ|yxZXkgR@q&SdCbVwOx2b~>D%BGQGB2Qh!?t0IYiq%P^NoQ6tU4;6b-xt*E`4>G z6846zns^UXf)9b>bOs$QI|R%;n2F$s2*Fq8()nSa2Dk>=)^k3`2CQss8!o;zP_9Pv zs?=24uYOmjP{xPKL6nq~$K|GCk&%%Vb#=YK=>Ugd-Zgy1=d_~y;&5EU-oc?r?f%`n z?V!nafz_Y%y1+g?717WS?;WyF3>*0KN50-;3{*HrtAx*P_C^2o!&&fi*1xN80C3}O zhxa)f);>HVyPc+@)nrJno`mCWMX^2#yN)uKS4(dq*3p@yLgQ8)tVRaIum1 z5wJ5c2>>WaQ%cOQ3Pn=HJyDhQ@My5`S^JpJ8YmhbB z5|XBu@TLO0l5R2d`!}_kni}qQaGx_D>%)nFu(10#f*g8&c{~@BmS$Z6bKOFJR1Yz+ zYgqRG5}lBc?<9$tDADf3U|MFEuJzx3;9_%*OQn-sURk7l2o2+81Nq1V>GaNK+!3Ljp<0!j#I#okB?6hb72=C9k>fqL@8gQGu@^6=S7z z-)Pa$PcSL7&3@rke>&$)O8gb#MN`khJa$uRIZUHVKzVt&#ozOtw%#`^-s7IT^HTZ= z=*LFmg=(?dEg>6=0GP$S-!xEG z9taVw(3at5#B{YK%jpboT3I&+gQ3lNofL)C!LbDhEM> z5W!0Rh%L)8`j7!VkS$n=t%sFqH`gk7WgsgfqcH2PO&J&%s8a>U*DBRv@+!sCt+!uj zBN}N?+UQaTMxcME7XyM`uyLVBa;&mANfX23@a$RdgEzmgNmiGaf&F-$73u5*v@(Fu zEr%dN+8`kMaKwr&fhYhCRb0wP#3&IC1pKXfClSSW@8Y*wZ(0Wg1pN$hME{BVhYKO& zlmCC*pLy`5cj^RGfdnLg2m-7V0Kt0~2P2cGE|lE*FM&Nq9L@wE{jJ|6ZD*=MWSmrx$ZBJVZ3Nv^caz3O`->^GD%| z`K>lk7CpeNkW+`rQ*zfmV$K%x0tX9fK(kDWCYYPO2Y^t>TM7AGQVkt!aVOy^$VZ7i zN-Yizy~n9j(T=uxkjxqcXJ#(`R$9tAm?3GnbKsql9^qS{x|H3eLJ*Xop&Mcz8%I0S zXlYPuiwT|cJs-OH5Z!-&C=sMtcYx#xn_U$JFb!+F3XLk)m#T>=o3U~*_mHWJdr}FA z?kQv~Og3lKLtii^?>#vE%l0fPE9?34OoL2!;ObG}yLav^T!cO<>EB0ia6B`1eA_6< zJFW+*o6}N4^{i^7R)xNRVCg;Zsy(&=eYy5-_xXdre}8oWV5_Ppda78SQGh05%lFni zsMx=JCGDHKUhKG}O)$YLCV89p`XJs?pFnE3sl6D%EKV&@0H-BfyTGl`dpVf9 z^3wTHH*XCNmj7^bKj6aWWqtU6kN=B0T=