---------------------------------------------------------------------- This is the API documentation for the spacypdfreader library. ---------------------------------------------------------------------- ## Functions The main entry point for converting a PDF into a spaCy Doc. ## Parsers Built-in PDF-to-text parsers. Pass one of these to the `pdf_parser` argument of `pdf_reader`, or bring your own. ---------------------------------------------------------------------- This is the User Guide documentation for the package. ---------------------------------------------------------------------- ## Guides ### Parsers Extracting text from PDF documents can be challenging. There are several different options in the python ecosystem. *spacypdfreader* makes it easy to extract text from PDF documents. At this time *spacypdfreader* has built in support for two options: - *pdfminer*: the default option ([GitHub](https://github.com/pdfminer/pdfminer.six) | [PyPi](https://pypi.org/project/pdfminer.six/) | [Docs](https://pdfminersix.readthedocs.io/en/latest/)) - *pytesseract*: alternative option ([GitHub](https://github.com/madmaze/pytesseract) | [PyPi](https://pypi.org/project/pytesseract/)) You can also bring your own custom PDF to text parser to use in *spacypdfreader*. ::: {.callout-tip} 💁‍♂️ Would you like to see another parser added? Please submit an issue on [GitHub](https://github.com/SamEdwardes/spaCyPDFreader/issues/new/choose) and the maintainer will look into adding support. ::: ::: {.callout-tip} Parsing big PDFs can be slow. For example, parsing a 166 page PDF document on an M1 mac took 166 seconds. If you are working with larger documents try breaking them into smaller documents and use multiprocessing. ::: ## Comparison of built in parsers All PDF to text parsers have their tradeoffs. The table below summaries the pros and cons of the built in parsers. | | *pdfminer* | *pytesseract* | | ------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | | When to use | ⚡️ When speed is more important than accuracy. | 🎓 When accuracy is more important than speed. | | Accuracy | 👌 **Medium:** from my experience *pdfminer* struggles with documents where the text is in one or more columns. | 👍 **High:** very good. Performs well on messy documents (e.g hand written text, PDFs with multiple columns of text on a single page). | | Speed | 👌 **Medium:** the text extraction is not instant, but it does not take forever. | 👎 **Slow:** the text extraction is very slow and will take hours on hundreds of pages. | | Installation | 👍 **Easy:** pure python, if you have installed *spacypdfreader* you already have everything you need. | 👎 **Complicated:** relies on additional non-python dependencies that can be complicated for beginners to install. | | How it works | Text is extracted directly from PDF using only Python. | Each pdf page is converted into an image. Optical character recognition is then run on each image. | ## *pdfminer* A pure Python library for extracting text from PDFs. **Installation** No action required, *pdfminer* will automatically be installed when you install *spacypdfreader*. **Usage** *pdfminer* is the default PDF to text extraction parser for *spacypdfreader*: ```python import spacy from spacypdfreader import pdf_reader nlp = spacy.load("en_core_web_sm") doc = pdf_reader("tests/data/test_pdf_01.pdf", nlp) ``` You could also be more verbose and pass in additional parameters. For a list of available parameters please refer to the pdfminer documentation for the [`extract_text`](https://pdfminersix.readthedocs.io/en/latest/reference/highlevel.html#extract-text) function. ```python import spacy from spacypdfreader import pdf_reader from spacypdfreader.parsers.pdfminer import parser nlp = spacy.load("en_core_web_sm") params = {"caching": False} doc = pdf_reader("tests/data/test_pdf_01.pdf", nlp, parser, **params) ``` ## *pytesseract* A PDf to text extraction engine that uses Googles *tesseract* OCR engine. **Installation** You can install most of the dependencies by pip installing *spacypdfreader* with some optional dependencies: ```bash pip install 'spacypdfreader[pytesseract]' ``` Unfortunately this will not always install all of the dependencies because some of them are non-python related. I find that installing pytesseract can be a little bit tricky for beginners. Please refer to [https://github.com/madmaze/pytesseract#installation](https://github.com/madmaze/pytesseract#installation) for details on how to install *pytesseract* if the above does not work. **Usage** To use *pytesseract* you must pass the *pytesseract* parser into the `pdf_parser` argument. For a list of available parameters you can pass in refer the documentation for the [`image_to_string`](https://github.com/madmaze/pytesseract) function from *pytesseract*. ```python import spacy from spacypdfreader import pdf_reader from spacypdfreader.parsers.pytesseract import parser nlp = spacy.load("en_core_web_sm") doc = pdf_reader("tests/data/test_pdf_01.pdf", nlp, parser) ``` ## Bring your own parser *spacypdfreader* allows your to bring your custom PDF parser. For examples of how to implement your own parser refer to: - , or - . To work with spacypdfreader a parser must be a function that: - Has an argument named `pdf_path`. - Has an argument named `page_number`. This argument should use *1 based indexing*. E.g. the value 1 refers to the first page of the PDF. - The function should return the text only for a single page of the PDF. This allows spacypdfreader to execute faster with multi-processing. ::: {.callout-warning} Version `0.3.0` changed how parsers are implemented. If you have created a custom parser that works with an older version of spacypdfreader it will need to be reimplemented. ::: ### Multiprocessing As of version `0.3.0` spacypdfreader has built in support for multi-processing. This can dramatically improve the time it takes to convert a PDF to text. ## Usage You can use multiprocessing with an parser. **pdfminer** ```python import spacy from spacypdfreader.spacypdfreader import pdf_reader nlp = spacy.load("en_core_web_sm") doc = pdf_reader("tests/data/test_pdf_01.pdf", nlp, n_processes=4) ``` **pytesseract** ```python import spacy from spacypdfreader.parsers import pytesseract from spacypdfreader.spacypdfreader import pdf_reader nlp = spacy.load("en_core_web_sm") doc = pdf_reader("tests/data/test_pdf_01.pdf", nlp, pytesseract.parser, n_processes=4) ``` ## Benchmark ```python import time from functools import wraps import spacy from spacypdfreader import pdf_reader from spacypdfreader.parsers import pytesseract def timeit(func): @wraps(func) def wrapper(*args, **kwargs): start = time.perf_counter() result = func(*args, **kwargs) end = time.perf_counter() print(f"Took {end - start:.6f} seconds to complete") return result return wrapper nlp = spacy.load("en_core_web_sm") file_name = "tests/data/wikipedia.pdf" @timeit def bench(n_processes): doc = pdf_reader(file_name, nlp, pytesseract.parser, n_processes=n_processes) return doc # With no multiprocessing bench(None) # Took 42.286371 seconds to complete # With multiprocessing bench(8) # Took 9.051591 seconds to complete ``` ### spaCy custom extensions When using [`pdf_reader`](../reference/spacypdfreader.pdf_reader.qmd) custom attributes and methods are added to spacy objects. ## `spacy.Doc` ### Extension attributes | Extension | Type | Description | | ------ | ------ | ------ | | `doc._.pdf_file_name` | `str` | The file name of the PDF document. | | `doc._.first_page` | `int` | The first page number of the PDF. | | `doc._.last_page` | `int` | The last page number of the PDF. | | `doc._.page_range` | `(int, int)` | The range of pages from the PDF. | | `doc._.page(int)` | `int` | Return the span of text related to the page. | ### Extension methods #### `Doc._.page` **Parameters:** | Name | Type | Description | Default | | ------------- | ----- | -------------------------------------------- | ---------- | | `page_number` | `int` | The PDF page number of the doc to filter on. | *required* | **Returns:** | Type | Description | | ------------ | -------------------------------------------------------- | | `spacy.Span` | The span of text from the corresponding PDF page number. | ## `spacy.Token` ### Extension attributes | Extension | Type | Description | | ------ | ------ | ------ | | `token._.page_number` | `int` | The PDF page number in which the token was extracted from. The first page is `1`. | ## Project ## 0.4.0 (2025-10-30) **Changes** - Support for Python 3.9 to Python 3.13 - Use `uv_build` back end instead of hatchling. **Fixes** None ## 0.3.2 (2024-10-04) **Changes** - Support for Python 3.8 to 3.12 and all future 3.0 versions of Python ([#16](https://github.com/SamEdwardes/spacypdfreader/issues/16), [#21](https://github.com/SamEdwardes/spacypdfreader/issues/21)) - Added local testing to test matrix of supported Python versions. - Switch from poetry to uv for managing project dependencies and building project. - Update dependencies. **Fixes** None ## 0.3.1 (2023-10-17) **Changes** - Support for `page_range` argument ([#16](https://github.com/SamEdwardes/spacypdfreader/issues/16), [#18](https://github.com/SamEdwardes/spacypdfreader/issues/18)). ```python import spacy from spacypdfreader import pdf_reader from spacypdfreader.parsers import pytesseract nlp = spacy.load("en_core_web_sm") doc = pdf_reader("tests/data/test_pdf_01.pdf", nlp, pytesseract.parser, n_processes=4, page_range=(2, 3)) ``` **Fixes** - Remove `shed` as a dependency. It was removing unused imports that were required ([#17](https://github.com/SamEdwardes/spacypdfreader/issues/17)). ## 0.3.0 (2023-05-17) **Changes** - Added support for multi-processing. For example: ```python import spacy from spacypdfreader.parsers import pytesseract from spacypdfreader.spacypdfreader import pdf_reader nlp = spacy.load("en_core_web_sm") doc = pdf_reader("tests/data/test_pdf_01.pdf", nlp, pytesseract.parser, n_processes=4) print(doc._.first_page) print(doc._.last_page) print(doc[12].text) print(doc[12]._.page_number) ``` - Changed the way in which parsers are implemented. They are now implemented with a function as opposed to a class. See for examples. **Fixes** None ## 0.2.1 (2022-01-09) - Added examples to the API docs. - Added continuous deployment for GitHub pages. ## 0.2.0 (2021-12-10) - Added support for additional pdf to text extraction engines: - [pytesseract](https://pypi.org/project/pytesseract/) - [textract](https://textract.readthedocs.io/en/stable/index.html) - Added the ability to bring your own pdf to text extraction engine. - Added new spacy extension attributes and methods: - `doc._.page_range` - `doc._.first_page` - `doc._.last_page` - `doc._.pdf_file_name` - `doc._.page(int)` - Built a new documentation site: [https://samedwardes.github.io/spaCyPDFreader/](https://samedwardes.github.io/spaCyPDFreader/) ## 0.1.1 (2021-12-10) - 0.1.1 Python ^3.7 support by @SamEdwardes in [https://github.com/SamEdwardes/spaCyPDFreader/pull/2](https://github.com/SamEdwardes/spaCyPDFreader/pull/2) ## Deployment checklist Before merging changes into main the following must be completed: - [ ] Bump the version number in _pyproject.toml_ and _spacypdfreader.\_\_init\_\__.py_ - [ ] Format the code: `just format` - [ ] Lint: `just lint` - [ ] Run pytest: ```bash just test-matrix just test-docs ``` - [ ] Test publishing to test PyPI: `just publish-test` - [ ] Check the docs locally: `just preview-docs` After merging the pull request: - [ ] Create a new release on GitHub - [ ] Publish latest package to PyPi: `just publish` The documentation is published automatically to GitHub Pages by the `CI Docs` GitHub Actions workflow on every push to `main`, so no manual docs deployment step is required. ## Code style The ruff code formatter should be run against all code. ```bash just format ``` ## Pre-commit hooks This project uses [prek](https://github.com/j178/prek) to manage Git pre-commit hooks. prek is a drop-in replacement for `pre-commit` and reads its configuration from `prek.toml`. The hooks run ruff (import sorting, linting, and formatting), rumdl (Markdown formatting), and a set of basic file checks. prek is included in the `dev` dependency group, so no separate install is required. Enable the hooks once after cloning the repository: ```bash uv run prek install ``` The hooks then run automatically on every `git commit`. To run them against all files on demand: ```bash uv run prek run --all-files ``` ## Documentation Documentation is built using [Great Docs](https://posit-dev.github.io/great-docs/), a Quarto-based documentation generator for Python packages. All of the documentation content lives in `great-docs.yml`, the `user_guide/` directory, and the docstrings within the package source code. Building the docs requires [Quarto](https://quarto.org/docs/get-started/) to be installed locally. Great Docs itself is run through `uvx`, so it does not need to be added to the project dependencies. ### Test the docs locally To preview the docs locally run the following command: ```bash just preview-docs ``` ### Publish the docs The docs are hosted using GitHub Pages at [https://samedwardes.github.io/spacypdfreader/](https://samedwardes.github.io/spacypdfreader/). They are rebuilt and published automatically by the `CI Docs` workflow (`.github/workflows/docs.yml`) on every push to the `main` branch.