Fix workflowsets imports, zero-coefficient labeling, and feature-cutoff bugs from #31 review - #40
Fix workflowsets imports, zero-coefficient labeling, and feature-cutoff bugs from #31 review#40eboyer221 wants to merge 2 commits into
Conversation
…ture-cutoff bugs from #31 review
amcim
left a comment
There was a problem hiding this comment.
Overall these changes look pretty good. I dealt with the merge conflict with the test file (did not require any real changes) by merging main into the branch, and I removed some more stale imports.
Some notes on the changes. In the original implementation of this in PR #31 , I aimed to recreate the results of VIP exactly. For (2), this is a good change. I copied how VIP did it, but its better this way and I approve of it.
Additionally for the open question here, the mismatch existed before PR #31. What this means is that the call to vip never used the tuned penalty. So if its changed to match, that would be changing the functionality.
For (3), I tested using demo_fit.rda data. With this change, its not just one missing feature that is kept. It also keeps features that have an importance score of 0. If we want to exclude the 0s we will need to add to the filter.
What
Addresses code review feedback on #31 (the vip::vi() replacement).
Three fixes, plus one item left as an open question:
Orphaned imports.
@importFrom workflowsets extract_fit_parsnipandextract_spec_parsnipare declared but nothing calls them -.viGlmnet()replaced the oldworkflowsets |> vip::vi()chain withparsnip::extract_fit_engine()directly. Removed both, and removedworkflowsetsfrom DESCRIPTION since nothing in the package uses it anymore.Zero coefficients mislabeled.
.viGlmnet()'sSign = ifelse(coefs > 0, "POS", "NEG")labeled an exact-zero coefficient "NEG" - indistinguishable from an actually-negative one. This matters for LASSO/elastic-net models, where zeroing out a coefficient is the normal way a feature gets excluded. Now labeled NA.Last feature silently dropped.
extractTopFeats()'s defaultprop_vi_top_feats = c(0, 1)is documented to return all features butfilter(cum_imp < cum_vi_upper & ...)used a strict<, and the least-important feature's cumulative importance is exactly equal to the total - so it always got excluded. Changed to<=.Left open
Whether
.viGlmnet()should use the tunedpenalty(as the multi-class path inextractTopFeats()already does) instead ofmin(glmnet_fit$lambda)- flagging for @AbhirupaGhosh.Testing
Added two tests to tests/testthat/test-core-ml.R. I confirmed both catch the original bugs by temporarily reverting the fix and re-running: the zero-coefficient test failed as expected, and testing against a real fitted model reproduced the exact symptom described - the least important feature (feat_noise_1) silently missing from the result. Full suite passes (214/214).