# What fills a data scientist’s Mac: environments, caches and datasets

Who it’s for · Updated 21 November 2026. By Mohit Nippanikar, who makes SiftMac.

## The short answer

- **Environments and data, in about equal measure.** A data scientist’s Mac fills with conda and Python environments, package caches, and copies of the same dataset: the download, the unzipped folder and the cleaned version.
- **What it costs you:** environments and caches come back from an `environment.yml` or `requirements.txt`. Datasets come back only if their source still has them.
- **Keep:** raw data you can’t download again, notebooks, and the files that rebuild each environment.

## 1. What fills a data scientist’s Mac

Every experiment leaves an environment, a cache and some data behind, and each tool keeps them in its own place:

| What | Where it lives | Comes back? | More |
| --- | --- | --- | --- |
| conda’s package cache | `pkgs` in your conda install, such as `~/miniconda3/pkgs` or `~/miniforge3/pkgs` | Yes, downloaded again | [conda](https://siftmac.com/safe-to-delete/conda-packages) |
| conda environments | `envs` in the same folder | From an `environment.yml`, if you kept one | [conda](https://siftmac.com/safe-to-delete/conda-packages) |
| Project virtual environments | `.venv` or `venv` in each project | From `requirements.txt`, `pyproject.toml` or a lockfile | [Python environments](https://siftmac.com/safe-to-delete/python-venv) |
| pip and uv caches | `~/Library/Caches/pip`, `~/.cache/uv` | Yes | [pip](https://siftmac.com/safe-to-delete/pip-cache), [uv](https://siftmac.com/safe-to-delete/uv-cache) |
| Hugging Face datasets | `~/.cache/huggingface/hub` for downloads, `~/.cache/huggingface/datasets` for Arrow files | Yes, while they’re on the Hub | [Hugging Face cache](https://siftmac.com/safe-to-delete/hugging-face-cache) |
| scikit-learn sample datasets | `~/scikit_learn_data` | Downloaded again | — |
| Raw downloads and their unzipped copies | Downloads, project `data` folders | Only from the source | [Downloads](https://siftmac.com/safe-to-delete/downloads-folder) |
| Notebook checkpoints and embedded outputs | `.ipynb_checkpoints`, plots saved inside `.ipynb` files | Checkpoints yes; outputs by running again | — |
| Experiment logs and model checkpoints | `mlruns`, `wandb`, saved weights | No: they record your runs | — |
| Compiled bytecode | `__pycache__` next to your modules | Yes, on the next run | [pycache folders](https://siftmac.com/safe-to-delete/pycache) |

Measure the shared parts, biggest last, then list your conda environments:

```
du -sh ~/miniconda3/pkgs ~/miniforge3/pkgs ~/anaconda3/pkgs ~/Library/Caches/pip ~/.cache/uv ~/.cache/huggingface ~/scikit_learn_data 2>/dev/null | sort -h
conda env list
```

And find data files over 500 MB in your usual folders (add your project folders to the list):

```
find ~/Documents ~/Downloads -type f -size +500M 2>/dev/null
```

## 2. What’s safe to clear, and what to keep

### Environments and caches

- **conda:** run `conda clean --all --dry-run` to see what would go, then again without `--dry-run`. It removes the index cache, unused packages, tarballs, temporary files and logs. Before removing packages, conda checks only the environments that `conda info --envs` lists, so make sure ones you created elsewhere with `--prefix` appear there.
- **Old environments:** run `conda env export > environment.yml` first if you might want one back, then `conda env remove -n <name>`.
- **pip and uv:** `pip cache purge` clears pip’s wheel and HTTP caches. `uv cache prune` removes only unused entries and `uv cache clean` everything; uv’s docs say never to change its cache by hand.
- **Virtual environments** in finished projects can go once their requirements are written down.

### Data: decide file by file

- Keep the raw data you can’t download again, and one copy of each cleaned dataset. The usual waste is the third copy: a `.zip`, the folder it unzipped to, and a converted Parquet file next to both.
- Hugging Face’s processed Arrow files are rebuilt from the downloads; from Python, `dataset.cleanup_cache_files()` removes one dataset’s cache files.
- Notebooks with many embedded plots grow large. `jupyter nbconvert --clear-output --inplace notebook.ipynb` strips the outputs; it overwrites the file, so try it on a copy first.
- Experiment folders like `mlruns` and `wandb` are the record of every run. Archive the ones you report on before deleting.

## 3. How SiftMac helps a data scientist

SiftMac’s [developer files](https://siftmac.com/features/developer-files) list a Python environment, whatever it’s called, when it has a `pyvenv.cfg` and a list of its packages beside it: `requirements.txt`, `pyproject.toml`, `environment.yml`, `uv.lock`, `poetry.lock`, `Pipfile` or `setup.py`. The project must be unchanged for 1, 3 or 6 months, and an environment a Python process is running from is skipped. conda environments in conda’s own folder aren’t listed.

- pip’s and Poetry’s caches in `~/Library/Caches` get the verdict **Rebuilds**. uv’s cache in `~/.cache/uv` isn’t cleaned by SiftMac; the [space map](https://siftmac.com/features/space-map) shows its size.
- Hugging Face **models** are listed one by one under [AI files](https://siftmac.com/features/ai-files) as You decide. Datasets in the same cache aren’t, so remove those with `hf cache rm`.
- [Duplicates](https://siftmac.com/features/duplicates) finds identical files of 1 MB or more in Desktop, Documents, Downloads, Movies, Music and Pictures and keeps the best copy. It skips any folder holding a `.git` and any Python environment, so data inside a repository isn’t compared.
- [Large & old files](https://siftmac.com/features/space-map) lists files of 1 GB or more, and those of 200 MB or more nobody has opened for a year, in the same folders.
- Old downloads lists a `.zip` in Downloads once its unzipped folder sits next to it, or once you opened it more than a week ago.
- Duplicates, large files and old downloads wait on the [Undo Shelf](https://siftmac.com/features/undo-shelf) for 7 days before the space is freed.

## 4. A cleanup routine for a data scientist’s Mac

1. Write down every environment you need: `conda env export`, `pip freeze > requirements.txt` or `uv lock`.
2. Run `conda clean --all`, after a dry run.
3. Remove conda environments and virtual environments of finished projects.
4. Clear the pip and uv caches.
5. Sort data: keep raw and final versions, delete intermediate copies and unzipped duplicates.
6. Remove Hugging Face datasets you no longer load.
7. Archive the experiment folders you report on to an external drive, then remove them locally.

## Questions

### Is it safe to delete conda’s pkgs folder by hand?

Better not. Environments can link back to packages in it, and deleting by hand can break them. `conda clean --all` removes only what no listed environment uses.

### Can I delete .ipynb_checkpoints folders?

Yes. They’re Jupyter’s autosaved copies of your notebooks; deleting them only removes the option to revert to the last checkpoint. They’re usually small, so they’re rarely the space problem.

### Why doesn’t SiftMac find duplicates in my data folders?

If the data lives inside a git repository, SiftMac skips the whole repository on purpose, because files that look identical in projects often belong there. Data in ordinary folders under Documents, Downloads or Desktop is compared.

## How SiftMac handles it

SiftMac lists Python environments in projects you haven’t changed for months when the file that rebuilds them is present, lists Hugging Face models one by one, and finds duplicate and large data files in your own folders, sending those to the Undo Shelf first.

Download: https://siftmac.com/SiftMac.dmg (scanning is free, and so is the first 2 GB of cleaning).

## Sources

- [conda: conda clean](https://docs.conda.io/projects/conda/en/latest/commands/clean.html)
- [pip: caching](https://pip.pypa.io/en/stable/topics/caching/)
- [uv: the cache](https://docs.astral.sh/uv/concepts/cache/)
- [Hugging Face Datasets: cache management](https://huggingface.co/docs/datasets/cache)
- [scikit-learn: get_data_home](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.get_data_home.html)
- [nbconvert: configuration options (clear-output)](https://nbconvert.readthedocs.io/en/latest/config_options.html)

## Related

- [Is it safe to delete conda’s pkgs folder and old environments?](https://siftmac.com/safe-to-delete/conda-packages)
- [Is it safe to delete a Python virtual environment (.venv)?](https://siftmac.com/safe-to-delete/python-venv)
- [What fills a Mac that runs local models and AI coding tools](https://siftmac.com/for/ai-developers)
- [How to find and remove duplicate files on a Mac](https://siftmac.com/guides/find-duplicate-files-on-mac)
- [Developer files: what SiftMac clears, and how it knows it comes back](https://siftmac.com/features/developer-files)

Canonical: https://siftmac.com/for/data-scientists
