Tracking issue for cleaning up python lib formulae

[cho-m]

A dedicated tracker for any tasks to handle based on decision in #157500.

For Homebrew/core:

• Triage python library formulae:
  • Remove python library formulae that do not meet our acceptable requirements and have low number of on-request installs
  • Deprecate python library formulae that do not meet our acceptable requirements and have high number of on-request installs (or have been around for a while in Homebrew/core). We can do a 3mo deprecation + 3mo disable period.
  • Convert/revert Python application formulae that are installed to prefix to instead use virtualenvs
  • Document anything remaining as approved exceptions

Extras for Homebrew/brew:

• Improve audit checks, particularly for --new formulae, e.g. audit: check for Python-wide site-package usage brew#16663
• Improve brew doctor and --ignore-dependencies "recommendation" for removed formulae to avoid users blindly running commands
• Optional: Maybe perform automatic "migration" to new rebuild/revision for removed formulae (which would also need to handle tab of recursive dependents)

---

Quick table based on 90-day install-on-request analytics data for formulae that may expose libs in HOMEBREW_PREFIX/lib. Also added whether the formula provides any files in bin and non-pure python libs.

Formula	90-day installs	90-day rank	# direct usage	Comment	bin/* cmd?	Ext libs
pycparser	38,795	136	4	dependency of cryptography	❎	❎
numpy	38,584	138	44	keep, Homebrew/brew#16662	✅︎	✅︎
cffi	33,997	154	7	dependency of cryptography	❎	✅︎
mercurial	16,247	273	4	keep	✅︎	✅︎
pillow	14,684	293	21	keep	❎	✅︎
pyqt@5	10,765	352	5	keep	✅︎	✅︎
pyqt	10,336	363	3	keep	✅︎	✅︎
python-setuptools	7,381	463	87		❎	❎
sip	6,932	477	6		✅︎	❎
cryptography	6,285	511	69	keep, Homebrew/brew#16662	❎	✅︎
scipy	6,248	512	4	keep, Homebrew/brew#16662	❎	✅︎
six	2,887	806	5		❎	❎
python-matplotlib	2,823	820	0		❎	✅︎
python-tabulate	1,771	1083	0		✅︎	❎
certifi	1,707	1109	164	keep, Homebrew/brew#16662	❎	❎
python-lxml	1,648	1135	1		❎	✅︎
python-packaging	1,547	1189	3		❎	❎
pyyaml	1,319	1293	3		❎	✅︎
python-markdown	1,293	1313	0		✅︎	❎
python-build	1,564	1176	1	reverted to venv, #165713 Decide if rename?	✅︎	❎
pymol	964	1577	0		✅︎	✅︎
pygit2	904	1630	0		❎	✅︎
pymupdf	824	1717	1		✅︎	✅︎
python-argcomplete	813	1731	0		✅︎	❎
lit	678	1908	1		✅︎	❎
wxpython	639	1962	0		✅︎	✅︎
python-ply	559	2101	1		❎	❎
python-chardet	494	2260	0		✅︎	❎
pyqt-builder	214	3530	3		✅︎	❎
ly	58	6025	0	keep bin, decide on lib.	✅︎	❎

Table for completed tasks (i.e. merged PRs), excluding removals:

Formula	90-day installs	90-day rank	# direct usage	Decision/PR	bin/* cmd?	Ext libs
docutils	32,666	162	10	reverted to venv, #168183	✅︎	❎
python-cryptography	24,100	205	0	renamed to cryptography	n/a	n/a
python-requests	4,118	650	0	deprecated, #166056	❎	❎
python-certifi	2,262	928	0	renamed to certifi	n/a	n/a
python-pytz	1,104	1447	0	deprecated, #168071	❎	❎
python-typing-extensions	1,033	1499	1	deprecated, #168114	❎	❎
python-urllib3	877	1667	1	deprecated, #166056	❎	❎
asciidoc	696	1886	25	reverted to venv, #165317	✅︎	❎
python-dateutil	644	1959	0	deprecated, #168071	❎	❎
python-psutil	603	2019	0	deprecated, #168071	❎	✅︎
python-trove-classifiers	507	2224	0	deprecated, #168181	❎	❎
python-idna	441	2385	1	deprecated, #166056	❎	❎
python-charset-normalizer	218	3503	1	deprecated, #166056	❎	❎

Collapsed table for removed formulae

Formula	90-day installs	90-day rank	# direct usage	Decision/PR	bin/* cmd?	Ext libs
python-markupsafe	1,742	1093	0	removed, #165568	n/a	n/a
python-platformdirs	447	2369	0	removed, #163603	n/a	n/a
python-boto3	398	2511	0	removed, #163229	n/a	n/a
python-pyparsing	389	2541	0	removed, #168201	❎	❎
python-jinja	366	2631	0	removed, #165495	n/a	n/a
python-hatchling	339	2749	0	removed, #166358	n/a	n/a
python-distlib	247	3277	0	removed, #163570
meson-python	235	3363	0	removed, #166060	❎	❎
python-setuptools-scm	179	3879	0	removed, #168073	❎	❎
python-flit-core	174	3942	0	removed, #168201	❎	❎
python-click	171	3983	0	removed, #165496
python-toml	170	4000	0	removed, #165493
python-sympy	169	4010	0	removed, #165494
python-mako	169	4016	0	removed, #165568
python-pluggy	149	4239	0	removed, #166358
python-pycurl	148	4247	0	removed, #164368
python-mutagen	116	4715	0	removed, #164365
python-pathspec	114	4742	0	removed, #166358
python-brotli	107	4869	0	removed, #165112
python-attrs	102	4937	0	removed, #163625
python-networkx	95	5069	0	removed, #166185
python-botocore	84	5306	0	removed, #163229
python-magic	81	5374	0	removed, #164158
python-filelock	78	5446	0	removed, #164940
python-openapi3	71	5638	0	removed, #163442
python-cli-helpers	70	5666	0	removed, #163436
python-colorama	69	5694	0	removed, #163437
python-distro	67	5750	0	removed, #164160
python-cycler	64	5839	0	removed, #166060	❎	❎
python-abseil	62	5908	0	removed, #163343
python-regex	61	5918	0	removed, #164146
python-docopt	55	6100	0	removed, #164149
python-pbr	55	6134	0	removed, #164154
python-configargparse	52	6242	0	removed, #163438
python-hatch-vcs	52	6243	0	removed, #164144
python-json5	52	6244	0	removed, #163642
python-xlsxwriter	51	6288	0	removed, #163646
python-kiwisolver	49	6382	0	removed, #166060	❎	✅︎
python-requests-oauthlib	48	6421	0	removed, #163590
python-hatch-fancy-pypi-readme	47	6462	0	removed, #164085
python-prompt-toolkit	47	6463	0	removed, #163576
python-termcolor	46	6511	0	removed, #163378
python-rich	45	6557	0	removed, #163623
python-asn1crypto	43	6655	0	removed, #163363
python-oauthlib	43	6656	0	removed, #163712
python-cachetools	41	6745	0	removed, #163425
python-pyproject-hooks	39	6835	0	removed, #166183
python-websocket-client	39	6836	0	removed, #163645
python-dicttoxml	32	7219	0	removed, #163567
python-mpmath	32	7220	0	removed, #165494
python-markdown-it-py	30	7362	0	removed, #164086
python-anytree	29	7431	0	removed, #163424
python-wcwidth	28	7489	0	removed, #163592
python-msgpack	27	7555	0	removed, #164148
python-jmespath	13	9027	0	removed, #163229
python-configobj	7	10540	0	removed, #163565
python-mdurl	7	10541	0	removed, #164086
python-s3transfer	6	10927	0	removed, #163229

[carlocab]

Should probably keep pygit2 -- it links with libgit2, and vendoring it into its dependents leads to really long build times (due to revision bumps) when libgit2 is updated.

[LecrisUT]

I've opened an issue to point out a potential problems with the build process, specifically affecting python projects which have setuptools-scm as a dependency