Blog
We are excited to announce the release of PyNIF3D – an open-source PyTorch-based library for research on neural implicit functions (NIF)-based 3D geometry representation. PyNIF3D aims to accelerate research by providing a modular design that allows for easy extension and combination of NIF-related components, as well as readily available paper implementations and dataset loaders.
The project can be found at https://github.com/pfnet/pynif3d. Please follow the installation steps described on the main page or feel free to contact us for further information.
Features
PyNIF3D provides a modular design which can be categorized into three main components: sampling, decoding and aggregation. Scene sampling refers to any method that samples pixels from an input image, rays that are cast from a 2D camera plane to a 3D environment or feature maps. Decoding refers to any NIF-based architecture that transforms the sampled data into some predictions, such as pixel values or occupancies. Aggregation refers to any method that aggregates those predictions in order to output the final values corresponding to the rendered image.
In the future we plan to support a fourth component, which allows for the visualization of intermediate training/inference steps in various representation formats (images, point clouds etc.).
Currently, the following features are available:
- Decoupled structure for NIF-based training and inference:
- Sampling functionalities (ray, pixel, features)
- Encoders (positional, Fourier)
- Efficient NIF model renderer with generic chunking
- Aggregation functionalities to generate final pixel/occupancy value
- Support for common datasets:
- Implementations of algorithm pipelines:
- NeRF (Mildenhall et al., 2020)
- Convolutional Occupancy Networks (Peng et al., 2020)
- Implicit Differentiable Renderer (Yariv et al., 2020)
- Other functionalities
- Generic layer generation with bias and weight initializers
- Explanatory exceptions and exception messages
- Detailed logging structure through decorators
- Ready-to-use pretrained models
- We have confirmed that the performance values of the current implementations are close to the values reported in the papers. However, since some algorithms are not completely reproduced, there may be some slight differences in terms of performance.
- Extensive API documentation and tutorials
Future plans
We plan to extend PyNIF3D with more components that can be tailored for the research task at hand, more paper implementations, support for common datasets and pretrained models. Your contribution will be greatly appreciated and we kindly ask you to read the contributing guidelines before submitting a PR.
Acknowledgements
This project is a collaboration between Preferred Networks Inc., Woven Core Inc., and Toyota Research Institute (TRI).
