Apple releases CoreNet, a library for training deep neural networks
source link: https://github.com/apple/corenet
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
CoreNet: A library for training deep neural networks
CoreNet is a deep neural network toolkit that allows researchers and engineers to train standard and novel small and large-scale models for variety of tasks, including foundation models (e.g., CLIP and LLM), object classification, object detection, and semantic segmentation.
Table of contents
What's new?
- April 2024: Version 0.1.0 of the CoreNet library includes
- OpenELM
- CatLIP
- MLX examples
Research efforts at Apple using CoreNet
Below is the list of publications from Apple that uses CoreNet:
Installation
You will need Git LFS (instructions below) to run tests and Jupyter notebooks (instructions) in this repository, and to contribute to it so we recommend that you install and activate it first.
On Linux we recommend to use Python 3.10+ and PyTorch (version >= v2.1.0), on macOS system Python 3.9+ should be sufficient.
Note that the optional dependencies listed below are required if you'd like to make contributions and/or run tests.
For Linux (substitute apt
for your package manager):
sudo apt install git-lfs
git clone [email protected]:apple/corenet.git
cd corenet
git lfs install
git lfs pull
# The following venv command is optional, but recommended. Alternatively, you can create and activate a conda environment.
python3 -m venv venv && source venv/bin/activate
python3 -m pip install --editable .
To install optional dependencies for audio and video processing:
sudo apt install libsox-dev ffmpeg
For macOS, assuming you use Homebrew:
brew install git-lfs
git clone [email protected]:apple/corenet.git
cd corenet
cd \$(pwd -P) # See the note below.
git lfs install
git lfs pull
# The following venv command is optional, but recommended. Alternatively, you can create and activate a conda environment.
python3 -m venv venv && source venv/bin/activate
python3 -m pip install --editable .
To install optional dependencies for audio and video processing:
brew install sox ffmpeg
Note that on macOS the file system is case insensitive, and case sensitivity
can cause issues with Git. You should access the repository on disk as if the
path were case sensitive, i.e. with the same capitalization as you see when you
list the directories ls
. You can switch to such a path with the cd $(pwd -P)
command.
Directory Structure
This section provides quick access and a brief description for important CoreNet directories.
Description | Quick Access |
---|---|
Getting Started | └── tutorials ├── train_a_new_model_on_a_new_dataset_from_scratch.ipynb ├── guide_slurm_and_multi_node_training.md ├── clip.ipynb ├── semantic_segmentation.ipynb └── object_detection.ipynb |
Training Recipesprojects/ directory.
Publication project directories generally contain the following contents:
| └── projects ├── byteformer ├── catlip (*) ├── clip ├── fastvit ├── mobilenet_v1 ├── mobilenet_v2 ├── mobilenet_v3 ├── mobileone ├── mobilevit ├── mobilevit_v2 ├── openelm (*) ├── range_augment ├── resnet └── vit |
MLX ExamplesREADME.md file within the corresponding example directory.
| └──mlx_example ├── clip └── open_elm |
Model ImplementationsEach model class is decorated by a
| └── corenet └── modeling └── models ├── audio_classification ├── classification ├── detection ├── language_modeling ├── multi_modal_img_text └── segmentation |
Datasets | └── corenet └── data └── datasets ├── audio_classification ├── classification ├── detection ├── language_modeling ├── multi_modal_img_text └── segmentation |
Other key directories | └── corenet ├── loss_fn ├── metrics ├── optims │ └── scheduler ├── train_eval_pipelines ├── data │ ├── collate_fns │ ├── sampler │ ├── text_tokenizer │ ├── transforms │ └── video_reader └── modeling ├── layers ├── modules ├── neural_augmentor └── text_encoders |
Maintainers
This code is developed by Sachin, and is now maintained by Sachin, Maxwell Horton, Mohammad Sekhavat, and Yanzi Jin.
Previous Maintainers
Contributing to CoreNet
We welcome PRs from the community! You can find information about contributing to CoreNet in our contributing document.
Please remember to follow our Code of Conduct.
License
For license details, see LICENSE.
Relationship with CVNets
CoreNet evolved from CVNets, to encompass a broader range of applications beyond computer vision. Its expansion facilitated the training of foundational models, including LLMs.
Citation
If you find our work useful, please cite the following paper:
@inproceedings{mehta2022cvnets,
author = {Mehta, Sachin and Abdolhosseini, Farzad and Rastegari, Mohammad},
title = {CVNets: High Performance Library for Computer Vision},
year = {2022},
booktitle = {Proceedings of the 30th ACM International Conference on Multimedia},
series = {MM '22}
}
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK