Machine learning is opening new approaches to build complex high-dimensional multiscale models in physics. Deep convolutional networks have obtained spectacular results for image, audio and bio-medical applications. We introduce a simplified architecture with wavelet scattering transforms. It provides a mathematical and algorithmic approach to approximate non-ergodic and non-Gaussian processes from few observations. Applications will be demonstrated on image textures, turbulent flows and Ising models. We also show that complex physical functionals can be regressed with such architectures. This is applied to the calculation of quantum molecular energies with no knowledge of quantum physics.