PurlDB: purl2sym Data collection Batch5

[pombredanne]

Here we add purl2sym support for a list of PURLs listed below.

1. Add support in FetchCode and PurlDB
2. Test of end-to-end support in the PurlDB

The PURLs are:

• pkg:gnu/libiconv
• pkg:generic/toybox
• pkg:generic/dropbear
• pkg:github/python/cpython
• pkg:generic/syslinux
• pkg:github/rpm-software-management/rpm
• pkg:github/pupnp/pupnp
• pkg:generic/miniupnpc
• pkg:generic/miniupnpd
• pkg:generic/minissdpd
• pkg:github/google/brotli
• pkg:github/libbpf/bpftool
• pkg:github/sqlite/sqlite
• pkg:generic/barebox
• pkg:github/nixos/nix
• pkg:generic/linux (this was moved to batch 6 and back here as this was pending resolution of PurlDB-requested package index scans can fail for larger packages with large scans and many scans at once scancode.io#1143 and The indexing of (very) large packages never completes #362 before we could do end-to-end test for linux.)

[keshav-space]

Completed in:

• purl2sym: Add metadata support for packages hosted on GitHub fetchcode#112
• Use authenticated requests for GitHub REST API calls fetchcode#115
• purl2sym: Support indexing of Batch3, Batch4 and Batch5 packages #360

The simple way to test this is to follow the below steps after installing PurlDB (https://github.com/nexB/purldb?tab=readme-ov-file#installation):

1. Go to /api/collect/ add PURL for indexing. ex: /api/collect/?purl=pkg:gnu/libiconv@1.17&addon_pipelines=collect_symbols_ctags
2. Once the indexing has completed go to /api/resources/and filter out resource based on PURL to see the symbols for each resource in extra_data field. ex: /api/resources/?purl=pkg:gnu/libiconv@1.17

Tip

Follow this tutorial to index the symbols and strings literals for a PURL/Package using various different addon pipelines. https://aboutcode.readthedocs.io/projects/PURLdb/en/latest/how-to-guides/tutorial_symbol_and_string_collection.html