Use mypy and ruff from pixi environment

[bouweandela]

Description

Use mypy and ruff from the pixi environment instead of installing them with pre-commit.

Pros

• More accurate mypy checks because all dependencies are installed
• Manage only one environment (pixi) instead of two (pixi and pre-commit)
• No unexpected test breaking because pre-commit does not use lock files for its environments as in Tests failing - github actions pre-commit, mypy - error: "TypeAliasType" not callable #3117
• mypy and ruff can be used without pre-commit. Useful if you want to run these tools with additional options, e.g. ruff check --fix --unsafe-fixes

Cons

• Does not work with pre-commit.ci
• Runs slower on HPC, I suspect this is because mypy is accessing more files

We could decide to move away from pre-commit.ci and instead just run the pre-commit hooks on CircleCI as part of the test suite. That would also have the advantage that it has one less CI service, so easier to understand for new contributors.

Link to documentation: https://esmvaltool--3124.org.readthedocs.build/projects/ESMValCore/en/3124/contributing.html#code-quality

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🧪 and 🛠 Documentation is available
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.48%. Comparing base (7e2f106) to head (f04b445).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3124   +/-   ##
=======================================
  Coverage   96.48%   96.48%           
=======================================
  Files         282      282           
  Lines       16407    16409    +2     
=======================================
+ Hits        15830    15832    +2     
  Misses        577      577           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[valeriupredoi]

thanks bud - a very welcome approach indeed! Approving preemptively (don't pull a Klaus on me since I approve drafts 😆 ) since I'll be away most next week

[valeriupredoi]

gotta be a solution to have pre-commit.ci work - let me dig around for a bit tomorrow

[valeriupredoi]

hey @bouweandela we has pixi in the pre-commit CI environment! Unfortunately that pixi is newer than the pixi we have in the box, so it says we should pixi migrate - I'll leave it to you if you want to accept this solution, or to revert to the non-pixi in CI env way. It took me a while to get to this, the pre-commit CI funcs are not nicely documented, but I think we have a good workaround now 🍺 Enjoy your holidays 🏖️

[bouweandela]

Thanks for having a go V! It looks like this installs the unrelated pixi package from PyPI. While it is probably possible to update the pre-commit hooks so they silently install pixi if it is not available locally by writing some script, I'm not convinced this would be doing our users a service. It may be more clear to just fail with the message that pixi is not installed, as they'll need to install it anyway in order to run ESMValTool.

[schlunma]

Thanks Bouwe, I like the idea!

Tested this locally and on Levante.

Locally

First run: ~38s
Subsequent runs: ~4s

Levante

First run: ~2.25 min
Subsequent runs: ~26s

Locally, that waiting time is totally fine, but on Levante it feels very slow. I am not 100% sure if this is still acceptable or not. Not sure if this is related to the current maintenance though, but as far as I am aware /home should not be affected by it.

Do you know how long the cache is valid when running pre-commit manually? As far as I can tell, on CircleCI you configured it so that it's valid for a week.

[bouweandela]

Do you know how long the cache is valid when running pre-commit manually? As far as I can tell, on CircleCI you configured it so that it's valid for a week.

With the current setup, pre-commit creates its own virtual environments and those are stored in ~/.cache/pre-commit/. I'm not entirely sure how pre-commit computes the cache keys, but I imagine it's based on the package name and version number from .pre-commit-config.yaml. With the changes here, the pixi environment will be used, which needs to be installed only when pixi.lock changed w.r.t. the previous run. There is also a .mypy_cache directory, which contains cached mypy stuff, I'm not sure how that is invalidated.

[schlunma]

I am pretty sure that this is related to mypy, which is by far the slowest step.

[bouweandela]

I agree, it takes about 18 seconds to run mypy on Levante. There is the option of using the daemon to speed things up, I enabled that now, but only works for reruns.

We could also consider switching to another type checker, see https://pydevtools.com/handbook/explanation/how-do-mypy-pyright-and-ty-compare/ for an overview. pyrefly seems an attractive option, it is a lot faster and has nice additional features such as integration in VSCode, so the type hint errors you see there are the same as what you see on the command line, and a command line tool to automatically add type hints.

[schlunma]

Yes, pyrefly looks very nice! I think we should give it a try.