$\newcommand{\kk}{\boldsymbol{k}} \newcommand{\eek}{\boldsymbol{e}_\boldsymbol{k}} \newcommand{\eeh}{\boldsymbol{e}_\boldsymbol{h}} \newcommand{\eez}{\boldsymbol{e}_\boldsymbol{z}} \newcommand{\cc}{\boldsymbol{c}} \newcommand{\uu}{\boldsymbol{u}} \newcommand{\vv}{\boldsymbol{v}} \newcommand{\bnabla}{\boldsymbol{\nabla}} \newcommand{\Dt}{\mbox{D}_t} \newcommand{\p}{\partial} \newcommand{\R}{\mathcal{R}} \newcommand{\eps}{\varepsilon} \newcommand{\mean}[1]{\langle #1 \rangle} \newcommand{\epsK}{\varepsilon_{\!\scriptscriptstyle K}} \newcommand{\epsA}{\varepsilon_{\!\scriptscriptstyle A}} \newcommand{\epsP}{\varepsilon_{\!\scriptscriptstyle P}} \newcommand{\epsm}{\varepsilon_{\!\scriptscriptstyle m}} \newcommand{\CKA}{C_{K\rightarrow A}} \newcommand{\D}{\mbox{D}}$
- LEGI group: Pierre Augier, A. Campagne, J. Sommeria, S. Viboud, C. Bonamy, N. Mordant (ERC Watu)...
- KTH group (Stockholm, Sweden): E. Lindborg, A. Vishnu, A. Segalini
- Diane Micard (LMFA)
Logiciels libres et Python pour l'acquisition, la gestion et le traitement de grosses données expérimentales, exemple d'une campagne sur la plate-forme Coriolis¶
MILESTONE project: studying mixing efficiency and stratified turbulence with¶
- experiments (need large scales to reach geophysical regimes $\Rightarrow$ Coriolis platform)
numerical simulations
open-source and open-science (Fluiddyn project)
Mixing efficiency and mixing coefficient¶
$$Ri_f \equiv \frac{\eps_P}{\eps_K + \eps_P} \quad \mbox{ and } \quad \Gamma \equiv \frac{\eps_P}{\eps_K},$$where $\eps_P$ and $\eps_K$ are the potential and kinetic energy dissipation.
Simple case: mixing a stably stratified fluid by stirring¶
stirring: injection of kinetic energy ($P_K$)
conversion from kinetic energy to potential energy (buoyancy flux) $- \mean{b w} = C_{K\rightarrow A} = \eps_P$
dissipation of kinetic and potential energy $\eps_K + \eps_P = P_K$
energy spent for mixing = dissipation of potential energy
- World Wide Web
- big data (storage, flux)
- companies (Youtube, Google)
- algorithms
- open-source (using in particular Python)
Open-source today¶
Big changes in how programs are developed:
- a lot of money (web, big companies as google or facebook and startups)
serious, good quality, good coding practices:
- distributed version control software (Git, Mercurial)
- forges for collaborative development (github, bitbucket, gitlab)
- issue tracker
- continuous integration
- online documentation
- unittests
- benchmarking
new tools and environments (for example Python)
Open-science¶
Transparency in scientific methods and results
Openness to full scrutiny
Ease of reproducibility
No more "reinventing the wheel" - particularly in code development
More practical:¶
Doing sciences with open-source methods and tools
Sharing using the web
Remark: different stories and tools for different communities¶
Fluid mechanics not in advance...
Dominance of Fortran and Matlab, which are
- good tools for few very specialized purposes
- but not plugged to the open-source and web dynamics
Bad averaged level in software engineering
Python language and its scientific ecosystem for open-science¶
Back to the research project:
MILESTONE project: studying mixing efficiency and stratified turbulence with¶
- experiments (need large scales to reach geophysical regimes $\Rightarrow$ Coriolis platform)
numerical simulations
open-source and open-science (Fluiddyn project)
Stratified flows and their different régimes¶
Introduction of 2 important non-dimensional numbers
- horizontal Froude number $F_h$
- buoyancy Reynolds number $\R = Re {F_h}^2$
Flow regimes in stratified fluids in the $[F_h,\ \R]$ parameter space¶
Methods: experiments in the Coriolis platform ($F_h \ll 1$ and $\R \gg 1$)¶
(13 m diameter)
The MILESTONE experiment¶
Stratified and rotating turbulence in the Coriolis platform
 2 sets of experiments
Summer 2016 (a collaboration between KTH, Stockholm, Sweden and LEGI): we tried many things.
Summer 2017: focused on mixing without rotation
Top view (MILESTONE 2016)
A new carriage for the Coriolis plateform!
- 3 m $\times$ 1 m
- runs on tracks (13 m long)
- good control in position ($\Delta x<$ 5 mm) and in speed ($U< 25$ cm/s)
Measurements: PIV and probes (density, temperature)
(in collaboration with Julien Salors, ENS Lyon)
Physical experiments can be seen as the interaction of autonomous physical objects
For the MILESTONE experiments:
- moving carriage, motor (Modbus TCP), position sensor (quadrature signal)
probes attached to a transverses (Modbus TCP)
scanning Particle Image Velocimetry (PIV):
- oscillating mirror driven by an acquisition board
- cameras triggered by a signal produced by an acquisition board
Issue: control with computers the interaction and synchronization of the objects
(in collaboration with Julien Salors, ENS Lyon)
Physical experiments can be seen as the interaction of autonomous physical objects
- In Python
- Object-oriented programming
- Very easy to write instrument drivers
- Automatic documentation for the instrument drivers
A little bit of Graphical User Interface is easy, fun and useful. We use PyQt.
Remark: reusable code, here, random movement for another experiment.
fluidimage: scientific treatments of images¶
(in collaboration with Cyrille Bonamy and Antoine Campagne, LEGI)
Many images (~ 20 To of raw data): embarrassingly parallel problem
- Open-source
- Clusters and PC, with CPU and/or GPU
Asynchronous computations
- topologies of treatments
- IO and CPU bounded tasks are splitted
Efficient algorithms and tools for fast computation with Python (Pythran, Theano, Pycuda, ...)
Images preprocessing
2D PIV, scanning PIV, stereo PIV, BOS
Utilities to display and analyze the PIV fields
- Plots of PIV fields (similar to PivMat)
- Calcul of spectra, anisotropic structure functions, characteristic turbulent length scales
Remark: highly inspired by PivMat (F. Moisy) and UVmat (J. Sommeria), that we continue to use, in particular for calibration
Calcul of scanning PIV on the LEGI cluster¶
Example of scripts to launch a PIV computation:
from fluidimage.topologies.piv import TopologyPIV
params = TopologyPIV.create_default_params()
params.series.path = '../../image_samples/Karman/Images'
params.series.ind_start = 1
params.piv0.shape_crop_im0 = 32
params.multipass.number = 2
params.multipass.use_tps = True
# params.saving.how has to be equal to 'complete' for idempotent jobs
# (on clusters)
params.saving.how = 'complete'
params.saving.postfix = 'piv_complete'
topology = TopologyPIV(params, logging_level='info')
topology.compute()
Calcul of scanning PIV on the LEGI cluster¶
Remark: launching computations on cluster is highly simplified by using fluiddyn:
from fluiddyn.clusters.legi import Calcul7 as Cluster
cluster = Cluster()
cluster.submit_script(
'piv_complete.py', name_run='fluidimage',
nb_cores_per_node=8,
walltime='3:00:00',
omp_num_threads=1,
idempotent=True, delay_signal_walltime=300)
Analysis, production of scientific figures, data sharing¶
- For one experiments, a lot of different files for different types of data (txt and hdf5 files)
- Classes for experiments and types of data (for example probe data or PIV field).
from fluidcoriolis.milestone17 import Experiment as Experiment17
iexp = 21
exp = Experiment17(iexp)
exp.name
print(f'N = {exp.N} rad/s and Uc = {exp.Uc} m/s')
print(f'Rc = {exp.Rc:.0f} and Fh = {exp.Fhc:.2f}')
print(f'{exp.nb_periods} periods of {exp.period} s')
print(f'{exp.nb_levels} levels for the scanning PIV')
Studying and plotting PIV data¶
from fluidcoriolis.milestone import Experiment
exp = Experiment(73)
cam = 'PCO_top' # MILESTONE16
# cam = 'Cam_horiz' # MILESTONE17
pack = exp.get_piv_pack(camera=cam)
piv_fields = pack.get_piv_array_toverT(i_toverT=80)
piv_fields = piv_fields.gaussian_filter(0.5).truncate(2)
piv = pack.get_piv2d(ind_time=10, level=1)
piv = piv.gaussian_filter(0.5).truncate(2)
piv.display()
_ = plt.xlim([-1.7, 0.5])
_ = plt.ylim([-1.3, 1.3])
Numerical simulations (pseudo-spectral)¶
fluidfft: unified API (C++ / Python) for Fast Fourier Transform libraries¶
- highly distributed, MPI (p3dfft, pfft, ...)
- CPU and GPU,
- 2D and 3D
fluidsim: framework to develop pseudo-spectral solvers (2D and 3D)¶
- highly modular (object oriented solvers)
- efficient (Cython, Pythran, mpi4py, h5py)
- inline data processing
- todo: operators and time stepping using Basilisk and Dedalus
Remarks: standardized benchmarks for spectral codes?
Conclusions on open science¶
Science in fluid mechanics with open-source methods and Python
Open-data: data in auto-descriptive formats (hdf5, netcdf, ...) + code to use and understand the data
A contribution to open-science, the fluiddyn project¶
open-source, clean, reusable codes (documented, tested, continuous integration)
- fluiddyn: base package containing utilities
- fluidlab: control of laboratory experiments
- fluidimage: scientific treatments of images (PIV)
- fluidfft: C++ / Python Fourier transform library (highly distributed, MPI, CPU/GPU, 2D and 3D)
- fluidsim: pseudo-spectral simulations in 2D and 3D
- fluidfoam: Python utilities for openfoam
- fluidcoriolis: running and analyzing experiments in the Coriolis platform
Main developpers:¶
Pierre Augier (LEGI), Cyrille Bonamy (LEGI), Antoine Campagne (LEGI), Ashwin Vishnu (KTH), Julien Salort (ENS Lyon)
Issues¶
- Collaborative dynamics? Adoption by the community? Developpers?
- Level in Python and coding in the community ! => Python training sessions in the lab and at university