AVRO-4232: [Python] Modernize the Python tooling by martin-g · Pull Request #3658 · apache/avro

martin-g · 2026-02-18T13:22:57Z

What is the purpose of the change

Use uv for management and ruff for linting and formatting

Verifying this change

This is mostly changes in Python package management (pyproject.toml and tox.ini)
The Python source files have small modifications caused by linting/formatting issues

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? See build.sh

Use `uv` for management and `ruff` for linting and formatting

Test 3.14 instead

python/mypy#20454

martin-g · 2026-02-18T13:43:29Z


 def main():
-    mail_server = http_server.HTTPServer(SERVER_ADDRESS, MailHandler)
+    mail_server = http.server.HTTPServer(SERVER_ADDRESS, MailHandler)


I have no idea how this worked before

Probably didn't. Is there evidence it was ever covered?

martin-g · 2026-02-18T13:44:25Z



 def looney_records():
-    return ({"first": f, "last": l, "type": t} for f, l, t in LOONIES)


Here for some reason there was a problem with l, so I renamed it to last

martin-g · 2026-02-18T14:24:34Z

@@ -1,84 +0,0 @@
-##


I am not sure whether this file should be removed.
But before removing it uv sync would fail with error like There is no "get" in NoneType or something like that.

It should be fine to remove this as long as the PEP 621 requirements are met. I haven't used uv that much (currently working in a php ecosystem), but I believe uv is fully compliant with that PEP.

It seems I have "translated" correctly most of the setuptools data (setup.cfg) to pyproject.toml (PEP 621).
My only uncertainty is package-data.

martin-g · 2026-02-18T14:54:30Z

@kojiromike I have some concerns about the removal of setup.cfg. I am not a Python dev and I am not fully confident in what I am doing here :-)
I'd be really thankful if you can test the dist target (uv run ./build.sh dist) and verify that things look good in the sdist and wheel packages.

martin-g · 2026-02-18T14:57:26Z

My main reason to introduce uv here is because the Python build started failing again without any changes in lang/py/. I guess some dependency has released a new version and to due not using a .lock file the build became unstable.

https://github.com/apache/avro/actions/runs/22087958485/job/63826627519

Policy by Infra: https://infra.apache.org/github-actions-policy.html > External actions Last allowed version: 7.1.6 (https://github.com/apache/infrastructure-actions/blob/2943971f47fadefa6d86af85c2c24c2cee262ee1/actions.yml#L25-L26) Co-authored-by: Ryan Skraba <ryan@skraba.com>

vallsv · 2026-03-29T11:08:24Z

                writers_schema = local_message.errors
                self.write_error(writers_schema, error, buffer_encoder)
-        except schema.AvroException as e:
+        except avro.errors.AvroException as e:


Hi.

This MR not only modernize the python tooling. But also fix that bug. Do you plane to update the library in pypi too?

Thanks a lot.

Right!
uv reported this problem and we fixed it!

It will be part of the 1.13.0 release but I cannot say when it will be made.

* AVRO-4232: [Python] Modernize the Python tooling Use `uv` for management and `ruff` for linting and formatting * Ignore uv.lock for ASFv2 licence check * Add librt as a dev dependency * Drop librt as dev-dep. It fails at CI with another error * Do not test Python 3.9 since it is not supported Test 3.14 instead * Try with mypy<1.19 python/mypy#20454 * Use uv for the Java interop tests too * Remove the flake8 settings. Now we use ruff * Update uv.lock * Set cwd to lang/py for `uv sync`. Do not use uv for C# scripts * Update the usage. Wrap variables in quotes * Move more entries from setup.cfg to pyproject.toml * Remove typechecks dependency. It seems to be tox specific * Do not shadow Python built-in `type` * Use `--frozen` for `uv sync` in CI * Update setup-uv Github action to v7 * Fix the package name for AvroException * First declare the local variable and then assign it a value * Use Python 3.10 for mypy * Move "scripts" from setup.cfg to pyproject.toml * Re-add the optional dependencies from setup.cfg to pyproject.toml * Re-add the package-data from setup.cfg to pyproject.toml * Use SHA for the external actions Policy by Infra: https://infra.apache.org/github-actions-policy.html > External actions Last allowed version: 7.1.6 (https://github.com/apache/infrastructure-actions/blob/2943971f47fadefa6d86af85c2c24c2cee262ee1/actions.yml#L25-L26) Co-authored-by: Ryan Skraba <ryan@skraba.com> * No need to call `build.sh` thru `uv run` --------- Co-authored-by: Ryan Skraba <ryan@skraba.com>

* AVRO-4232: [Python] Backport uv/ruff tooling to branch-1.12 Backports the Python tooling modernization (AVRO-4232, #3658) so that the Java interop workflow's 'uv sync --frozen' step works on branch-1.12. The workflow change was previously backported without the accompanying lang/py migration, leaving no uv.lock or [project] table, which caused 'uv sync --frozen' to fail. - Migrate lang/py from tox/setuptools to uv + ruff - Add uv.lock and [project]/[dependency-groups] to pyproject.toml - Remove tox.ini and setup.cfg - Update build.sh (ruff/coverage/mypy targets) - Convert test-lang-py.yml to uv (drops py3.9 CI, adds 3.14) - Exclude lang/py/uv.lock from RAT license check Version pinned to 1.12.2 and packages.find restricted to avro* to fix flat-layout package discovery. * AVRO-4232: [Python] Use pinned ruff (uv run) instead of uvx in build.sh uvx fetches the latest ruff, whose stricter default lint rules flag 252 pre-existing issues in the (not-yet-modernized) branch-1.12 source. Use 'uv run ruff' to run the version pinned in uv.lock (0.15.1), matching main and keeping lint reproducible.

github-actions Bot added build Python labels Feb 18, 2026

AVRO-4232: [Python] Modernize the Python tooling

6ec71fe

Use `uv` for management and `ruff` for linting and formatting

martin-g force-pushed the avro-4232-use-uv-ruff branch from 027ddcf to 6ec71fe Compare February 18, 2026 13:26

martin-g added 5 commits February 18, 2026 15:27

Ignore uv.lock for ASFv2 licence check

9902208

Add librt as a dev dependency

d81a4a7

Drop librt as dev-dep. It fails at CI with another error

a411989

Do not test Python 3.9 since it is not supported

9a2ff4e

Test 3.14 instead

Try with mypy<1.19

34eae25

python/mypy#20454

martin-g commented Feb 18, 2026

View reviewed changes

martin-g added 3 commits February 18, 2026 15:49

Use uv for the Java interop tests too

a427cfb

Remove the flake8 settings. Now we use ruff

d401b80

Update uv.lock

9e71d4b

martin-g requested a review from kojiromike February 18, 2026 13:50

Set cwd to lang/py for uv sync. Do not use uv for C# scripts

d73daa5

kojiromike reviewed Feb 18, 2026

View reviewed changes

Comment thread .github/workflows/test-lang-java.yml

martin-g commented Feb 18, 2026

View reviewed changes

kojiromike reviewed Feb 18, 2026

View reviewed changes

Comment thread lang/py/pyproject.toml

kojiromike approved these changes Feb 18, 2026

View reviewed changes

martin-g added 11 commits February 18, 2026 16:26

Update the usage. Wrap variables in quotes

cb2b731

Move more entries from setup.cfg to pyproject.toml

7023260

Remove typechecks dependency. It seems to be tox specific

7ff957b

Do not shadow Python built-in type

f246a65

Use --frozen for uv sync in CI

cdedb56

Update setup-uv Github action to v7

78f3a5c

Fix the package name for AvroException

e462824

First declare the local variable and then assign it a value

e0484b3

Use Python 3.10 for mypy

3c7c8c7

Move "scripts" from setup.cfg to pyproject.toml

aeb32b9

Re-add the optional dependencies from setup.cfg to pyproject.toml

c15c680

Re-add the package-data from setup.cfg to pyproject.toml

4724e89

RyanSkraba reviewed Feb 18, 2026

View reviewed changes

Comment thread .github/workflows/test-lang-py.yml Outdated

RyanSkraba reviewed Feb 18, 2026

View reviewed changes

Comment thread .github/workflows/test-lang-java.yml Outdated

RyanSkraba reviewed Feb 18, 2026

View reviewed changes

Comment thread .github/workflows/test-lang-py.yml Outdated

martin-g and others added 2 commits February 19, 2026 08:37

No need to call build.sh thru uv run

712fde3

martin-g merged commit 7ee89c2 into main Feb 23, 2026
27 checks passed

martin-g deleted the avro-4232-use-uv-ruff branch February 23, 2026 08:26

This was referenced Feb 23, 2026

AVRO-4232: Install uv for the Python docs build #3664

Merged

AVRO-4232: Install uv for the JavaScript & Docker CI workflows #3665

Merged

vallsv reviewed Mar 29, 2026

View reviewed changes

iemejia mentioned this pull request Aug 6, 2026

AVRO-4232: [Python] Backport uv/ruff tooling to branch-1.12 #3922

Merged

Navigation Menu

Uh oh!

AVRO-4232: [Python] Modernize the Python tooling - #3658

AVRO-4232: [Python] Modernize the Python tooling#3658
martin-g merged 24 commits into
mainfrom
avro-4232-use-uv-ruff

martin-g commented Feb 18, 2026

Uh oh!

martin-g Feb 18, 2026

Uh oh!

kojiromike Feb 18, 2026

Uh oh!

martin-g Feb 18, 2026

Uh oh!

Uh oh!

martin-g Feb 18, 2026

Uh oh!

kojiromike Feb 18, 2026

Uh oh!

martin-g Feb 19, 2026

Uh oh!

Uh oh!

martin-g commented Feb 18, 2026

Uh oh!

martin-g commented Feb 18, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vallsv Mar 29, 2026

Uh oh!

martin-g Mar 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants



		def looney_records():
		return ({"first": f, "last": l, "type": t} for f, l, t in LOONIES)

Uh oh!

Conversation

martin-g commented Feb 18, 2026

What is the purpose of the change

Verifying this change

Documentation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

martin-g commented Feb 18, 2026

Uh oh!

martin-g commented Feb 18, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants