Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Add proper typing @overload to `zimscraperlib.image.optimize_xxx` methods (#273)
- Backport wabac.JS change around rewriting: detect when JS is in 'strict' mode and avoid using 'arguments' (#286)
- Fix installation from source, which missed wombatSetup.js (#287)
- Fix obsolete `libjpeg8-dev` reference, replace with `libjpeg-dev` and clarify dev headers are only needed when building Pillow from source (#152)
- Improve contribution setup instructions: use `hatch shell`, clarify commands must be run from local clone root (#153)

### Changed

Expand Down
49 changes: 33 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,33 +33,32 @@ See documentation at [Read the Docs](https://python-scraperlib.readthedocs.io/)

# Dependencies

- libmagic
- wget
- libzim (auto-installed, not available on Windows)
- Pillow
- FFmpeg
- gifsicle (>=1.92)
- libcairo (if you use the image manipulation, this is used for svg conversion)
Most dependencies are installed automatically via PyPI. The following system packages may be required depending on which features you use:

- **libmagic** — required for file type detection (used in most scrapers)
- **wget** — required only for `zimscraperlib.download` functions
- **FFmpeg** — required only for video processing functions
- **gifsicle** (>=1.92) — required only for GIF optimization
- **libcairo** — required only for SVG-to-PNG conversion
- **libzim** — auto-installed via PyPI, not available on Windows
- **Pillow** — auto-installed via PyPI; pre-built wheels are used by default and no system image libraries are needed. Only if you need to build Pillow from source should you install additional system libraries — see [Pillow's build documentation](https://pillow.readthedocs.io/en/latest/installation/building-from-source.html) for details.

## macOS

```sh
brew install libmagic wget libtiff libjpeg webp little-cms2 ffmpeg gifsicle
brew install libmagic wget ffmpeg gifsicle
```

## Linux

```sh
sudo apt install libmagic1 wget ffmpeg \
libtiff5-dev libjpeg8-dev libopenjp2-7-dev zlib1g-dev \
libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk \
libharfbuzz-dev libfribidi-dev libxcb1-dev gifsicle
sudo apt install libmagic1 wget ffmpeg gifsicle
```

## Alpine

```
apk add ffmpeg gifsicle libmagic wget libjpeg
```sh
apk add ffmpeg gifsicle libmagic wget
```

# Contribution
Expand All @@ -68,11 +67,29 @@ This project adheres to openZIM's [Contribution Guidelines](https://github.com/o

This project has implemented openZIM's [Python bootstrap, conventions and policies](https://github.com/openzim/_python-bootstrap/docs/Policy.md) **v1.0.2**.

All instructions below must be run from the root of your local clone of this repository.

If you do not already have it on your system, install [hatch](https://hatch.pypa.io/latest/install/):

```shell
pip install hatch
pip install ".[dev]"
```

Start a hatch shell — this will install all dependencies including dev in an isolated virtual environment:

```shell
hatch shell
```

Set up the pre-commit Git hook (runs linters automatically before each commit):

```shell
pre-commit install
# For tests
```

Run tests with coverage:

```shell
invoke coverage
```

Expand Down