Currently, I think we are downloading all the packages every time CI is run. This takes a long time(~5-10 min. on azure) and makes CI flakier, since sometimes we can't connect to the Anaconda servers, and sometimes we timeout on Azure. We should cache the packages downloaded and only check for updates everytime the CI is run. I think we're doing some form of this on GHA, but we should also do it on azure.
cc @fangchenli
Comment From: fangchenli
It shouldn't be hard. https://docs.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops#pythonanaconda
I'll try cache for the windows build in #42581
Comment From: alimcmaster1
It shouldn't be hard. https://docs.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops#pythonanaconda
I'll try cache for the windows build in #42581
Do you still want to work on this? If not I think we can mark as good first issue/contrib welcome as instructions above look comprehensive :)
@fangchenli
Comment From: amrutha-mukkatira
1) Export yaml : conda env export > env.yaml and then if there is a section of pip: packages, try to replace them all with Conda versions (editing the YAML). If there isn't a pip section, then it might be the conda clean -p thing. Either way, recreate your environment using the YAML
2)conda env create -f env.yaml -n new_env This should force redownloading packages missing from the cache, as well as switch to non-PyPI versions of packages, assuming you removed all the pip: packages.
I would expect conda-pack to then work on this rebuilt version of the environment.
Comment From: Sara-cos
Is it still open?
Comment From: mroeschke
I think we do with now with provision-with-micromamba and don't use azure now so closing this out.