DFT with VASP. Band gaps, formation energies, elastic properties, phonons. Most widely used in ML-materials research.
Largest computational database. High-throughput DFT. Strong for alloys, intermetallics, and electronic structure.
Open Quantum Materials Database. Focus on formation enthalpies and thermodynamic stability. Built with VASP + PBE.
Crystallography Open Database. Experimental X-ray and neutron diffraction structures. CIF format. Open access.
Raw DFT calculation files from diverse codes (VASP, Quantum ESPRESSO, FHI-aims). Largest archive of calculation outputs.
Inorganic Crystal Structure Database. Commercial but widely licensed. Most comprehensive experimental crystal structure collection.
Every database on the previous slide is a pile of DFT calculations. Before we touch Python, it helps to know what actually produced those numbers: which code ran the calculation, and which settings it used. This is also what the "Reproducibility" point on the last slide was really about.
The most-used DFT code in materials databases. Paid licence (per institution). Plane-wave basis set with PAW pseudopotentials. Materials Project, AFLOW, and OQMD are all built on VASP.
Same plane-wave pseudopotential approach as VASP, but free to use. Popular in universities without a VASP licence. One of the 40+ codes NOMAD archives raw output from.
Each uses a different basis set (plane-wave vs. localised atomic orbitals). Results from different codes are not automatically comparable, even for the same material.
Core electrons barely affect chemistry but are expensive to simulate. A pseudopotential replaces them with a smooth effective field, leaving only the valence electrons to solve for explicitly.
DFT needs an approximation for electron-electron interaction. PBE (a GGA functional) is the near-universal default. It systematically underestimates band gaps by roughly 30 to 50 percent versus experiment.
+U corrects transition-metal oxides. k-point density and energy cutoff control the accuracy/speed trade-off.
Code split into cells you run one at a time, with output shown directly below each cell. Built for exploring data step by step. Saved as .ipynb files.
One continuous file that runs top to bottom in a single pass, with no cells and no inline output. Built for larger projects and automation. Saved as .py files.
Runs in your browser. Google provides the computer. Free GPU included. No install. Best starting point for this course.
Same cell format as Colab but runs on your own machine. Files stay local. JupyterLab is the modern version with a cleaner interface and file browser sidebar.
Open .ipynb notebooks or write .py scripts. Best for larger projects with multiple files.
Not a place to run code. Google Drive for code. Tracks every change. All course notebooks live here so you can copy and run them in Colab with one command.
Free notebook environment with public datasets and ML competitions. Great for practising after this course on real materials and chemistry problems.
Download pre-trained materials AI models in one line. Spaces hosts runnable web apps so you can test models without writing any code at all.
The brain of your computer. Handles a few complex tasks at a time very quickly. Every laptop has one. Perfect for classical ML (Random Forest, ElasticNet, linear models) on tabular materials data. Also the standard for DFT calculations: VASP and Quantum ESPRESSO are CPU-optimised.
Handles thousands of tiny calculations simultaneously. Originally built for video game graphics: now the engine of modern AI. Essential for training GNNs on crystal structures, ML potentials (MACE, CHGNet), and any deep learning on materials. Free GPUs available on Colab and Kaggle.
Google's custom chip built specifically for matrix operations in neural networks. Faster than GPU for very large models (billions of parameters). Rarely used directly in materials informatics today: most materials models are small enough that a GPU is sufficient. Available free on Colab.
A virtual environment is an isolated Python sandbox per project. Libraries installed in one project never conflict with another. Always use one.
# Step 1: create the environment python -m venv matenv # Step 2: activate it # Windows: matenv\Scriptsctivate # Mac / Linux: source matenv/bin/activate # Step 3: install your packages pip install mp-api pymatgen matminer # Step 4: deactivate when done deactivate
# Create environment with a specific Python version conda create -n matenv python=3.11 # Activate conda activate matenv # Install (conda first, then pip for extras) conda install numpy pandas scikit-learn pip install mp-api pymatgen matminer # List all environments conda env list
Ctrl+Shift+P (Windows) or Cmd+Shift+P (Mac)Python: Create Environment, press Enter, choose Venv or Condamatenv environment.pip install normally..ipynb file, click the kernel selector (top right), choose your matenv kernel.What is pip?
Python's package manager – like an app store. You type the name of a library, pip finds it on the internet, downloads it, and sets it up so Python can use it. You only need to install each library once per environment.
import library_name with no extra steps.Terminal (Windows / Mac / Linux)
pip install mp-api pymatgen matminer aflow optimade-client requests pandas numpy
Google Colab cell
# ! = system command, not Python code !pip install mp-api pymatgen matminer aflow optimade-client requests pandas numpy
Jupyter Notebook cell
!pip install mp-api pymatgen matminer aflow requests pandas numpy # Safer: installs into the exact Python Jupyter is using: import sys; !{sys.executable} -m pip install mp-api pymatgen
VS Code
# In the built-in terminal (Terminal → New Terminal): pip install mp-api pymatgen matminer aflow requests pandas numpy # Or use !pip inside a .ipynb notebook cell
pip install --upgrade numpy
pip install -r requirements.txt
pip install --no-cache-dir pymatgen
pip install -q matminer
pip install -e .
pip show numpy
pip list
pip freeze > requirements.txt
Six words you will hear constantly in this lecture, explained once, in plain terms.
A database is a library: millions of records on shelves you cannot browse yourself. A client is the librarian software that sits between you and it. You ask in simple Python; it handles the technical back-and-forth. mp-api is the librarian for the Materials Project.
Not every database has a librarian. Some just have a window in the wall: you slide a note through with your question, and a note comes back. requests is how Python writes and passes that note. Used for OQMD, COD, and NOMAD.
A crystal structure records exactly where every atom sits: positions, distances, angles, the repeating unit cell. Like a building's floor plan, but for atoms. It is the single most important thing a database hands back.
An object bundles data with the tools to use it, like a phone holding your contacts and also knowing how to call them. A Structure object holds atom positions and already knows how to compute density, volume, and symmetry.
An agreed way of writing structure data so any program can read it the same way, like every hospital filling in a patient form identically. CIF is that standard for crystal structures.
Not the natural environment: a collection of tools that work together for one purpose, like a carpenter's workshop. ASE is a workshop for scientists who simulate atoms. Installing libraries is building your own.
People new to this mix these four up constantly, because they all show up in the same sentence. The trick is to ask one question about each: what is it actually describing?
Structure answers "what material is this?": which atoms, where they sit. It is the real-world thing you are describing, not a piece of code. Object answers a completely different question, "how is this packaged in Python?": it is a general programming idea that has nothing to do with atoms specifically. A Structure happens to be one example of an object; so is a DataFrame, so is a trained model. File format answers "how is this written down and shared?": the exact same structure can be saved as a CIF, a POSCAR, or an XYZ file, the way the same recipe can be written in English or French. The format is not the thing itself, only one way of recording it. Environment answers a question that has nothing to do with atoms at all, "where am I typing this code?": Colab, VS Code, your own laptop. The same Structure, saved in the same CIF file, behaves identically no matter which environment opens it.
Takes the numbers from the Features layer and does the actual work with them: builds tables, trains a prediction model, and draws the charts you see in results.
A machine learning model cannot read a crystal structure. This layer converts one into a list of plain numbers, describing it the way you might describe a house by its rooms, area, and age.
Turns the raw data a database sends back into a proper crystal structure object: exactly where every atom sits, ready for Python to work with.
Every pipeline starts here: these tools fetch the raw data, band gaps, formation energies, atom positions, from a materials database over the internet and bring it into your Python code.
Read the stack from the bottom up: raw data comes in at the Databases layer, and each layer above turns it into something more useful, until Analysis produces an actual answer. Install everything at once:
pip install mp-api pymatgen matminer aflow requests pandas numpy
Foundation of all scientific Python; every other library on this slide is built on it. Inside it: the ndarray type (a fixed-size, same-dtype grid of numbers), numpy.linalg for matrix operations, and numpy.random for sampling. In materials work it stores lattice matrices, atom position vectors, and arrays of property values.
# NumPy is the library; "np" is just a short nickname for it, # so you don't have to type "numpy" every single time import numpy as np # store the three numbers 1.0, 2.0, 3.0 as one NumPy array, # so maths on all three at once is fast a = np.array([1.0, 2.0, 3.0]) # build a 3x3 grid of zeros: the same shape as a crystal's lattice matrix m = np.zeros((3, 3)) # the average of the three numbers stored in "a" np.mean(a) # how spread out those numbers are, on average, from that mean np.std(a) # multiplies the grid "m" by the list "a": this is how you rotate # or transform a set of atom coordinates using a lattice matrix np.dot(m, a)
Load, filter, merge, and export materials datasets: each row is one material, each column is one property. Inside it: the DataFrame (a full table) and Series (a single labelled column) classes, plus readers like read_csv, read_json, and read_excel.
# pandas is the table library; "pd" is its usual nickname import pandas as pd # open a spreadsheet-style CSV file and load it as a table (DataFrame) df = pd.read_csv("band_gaps.csv") # keep only the rows where the band_gap column is bigger than 1.0 df[df["band_gap"] > 1.0] # show only the "formula" and "band_gap" columns, hide the rest df[["formula", "band_gap"]] # a quick summary: how many rows, their average, spread, min and max df.describe() # save this table back to disk as a new CSV file df.to_csv("filtered.csv")
Random Forest, ElasticNet, cross-validation, scaling: every model follows the same import, fit, predict pattern. Inside it: sklearn.ensemble (tree-based models), sklearn.linear_model (linear/regularised models), sklearn.model_selection (splitting and cross-validation), and sklearn.preprocessing (scaling and encoding).
# RandomForestRegressor is one specific model; scikit-learn has many from sklearn.ensemble import RandomForestRegressor # a helper that automatically trains and tests a model several times from sklearn.model_selection import cross_val_score # rescales your features so they are all on a similar numeric range, # which many models need in order to learn well from sklearn.preprocessing import StandardScaler # create the model: 100 decision trees, not trained yet model = RandomForestRegressor(n_estimators=100) # show the model your training examples, so it can learn the pattern model.fit(X_train, y_train) # ask the trained model to guess values for materials it has not seen model.predict(X_test) # repeat train-then-test 5 times on different slices of the data, # so you get 5 scores instead of trusting just one lucky split cross_val_score(model, X, y, cv=5)
Query OQMD, COD, NOMAD: any database without its own Python client. Inside it: requests.get / requests.post for the two common HTTP verbs, and a Response object with .json(), .status_code, and .text. Every REST API here returns JSON, which .json() turns into a Python dictionary.
# requests is Python's library for talking to websites and APIs import requests # send a question to this web address and wait for the answer r = requests.get( "https://oqmd.org/oqmdapi/formationenergy", # extra options attached to the question: at most 10 results, # and only send back these two named fields params={"limit": 10, "fields": "name,delta_e"} ) # the number 200 means the request succeeded; other numbers mean an error r.status_code # turn the answer into a Python dictionary, then read its "data" part data = r.json()["data"]
Read API keys securely, create output folders, build cross-platform file paths. Inside it: os.environ (a dictionary of environment variables), os.makedirs for folders, and pathlib.Path, an object representing a file path with methods like .exists(), .stem, .suffix, and .glob().
import os from pathlib import Path # read a secret key that was set outside the code, never type # a real API key directly into a script anyone might see key = os.environ.get("MP_API_KEY") # create a folder called "results", and do nothing if it already exists os.makedirs("results", exist_ok=True) # builds a file path by joining "data" and "band_gaps.csv" correctly, # whether you are on Windows, Mac, or Linux p = Path("data") / "band_gaps.csv" # check whether that file actually exists on disk yet p.exists() # pull out just the file name ("band_gaps") and the extension (".csv") p.stem; p.suffix
Silences pymatgen/matminer deprecation messages so your notebook output stays readable. Inside it: filterwarnings takes an action ("ignore", "error", "default") and an optional warning category to target only one kind of warning.
import warnings # hide every warning message; blunt, but the most common approach warnings.filterwarnings("ignore") # a gentler option: hide only deprecation notices, so you still # see warnings that might point to a real problem warnings.filterwarnings( "ignore", category=DeprecationWarning )
Band gap distributions, parity plots, learning curves, and it renders inline in notebooks. Inside it: matplotlib.pyplot is the state-based interface most people use day to day, sitting on top of Figure and Axes objects for finer control over multi-panel plots.
# the plotting toolkit; "plt" is its usual nickname import matplotlib.pyplot as plt # draw a histogram: how many materials fall into each of 50 band-gap ranges plt.hist(df["band_gap"], bins=50) # one dot per material, comparing what actually happened to what # the model predicted; dots on a diagonal line mean a good model plt.scatter(y_test, y_pred) # label the horizontal axis so readers know what it shows plt.xlabel("Actual (eV)") # label the vertical axis the same way plt.ylabel("Predicted (eV)") # actually draw the finished chart on screen plt.show()
Wraps any loop and shows a live count, percentage, and time estimate; essential once you are featurizing thousands of structures. Inside it: tqdm() for scripts and the terminal, tqdm.notebook.tqdm() for an animated bar in Colab/Jupyter, and tqdm.pandas() to add a bar to a pandas .apply() call.
# this version is for scripts and the terminal from tqdm import tqdm features = [] # wrapping "structures" in tqdm() changes nothing about the loop # itself, it just adds a progress bar while it runs for s in tqdm(structures): # the bar's count ticks up by one each time this line runs features.append(featurize(s)) # inside a notebook (Colab, Jupyter), use this version instead, # it draws a nicer, animated bar from tqdm.notebook import tqdm
json for human-readable data files that any language can read; pickle for saving trained models so you never retrain from scratch. Inside it: json.dump/load handle text and only basic Python types; pickle.dump/load handle binary data and almost any Python object, including a fitted model, but should never load a file from an untrusted source.
import json, pickle # save a dictionary or list to a text file anyone can open and read json.dump(results, open("out.json", "w")) # load that same file back in as a Python dictionary or list json.load(open("out.json")) # save your trained model to disk, so tomorrow you can reuse it # instead of training it all over again; "wb" means write binary pickle.dump(model, open("model.pkl", "wb")) # load that saved model back in, ready to use; "rb" means read binary model = pickle.load(open("model.pkl", "rb"))
Periodic table data in Python: any element property, electronegativity, atomic radius, ionisation energy, electron configuration, density, complementing MAGPIE for hand-crafted materials features. Inside it: the element() function returns an Element object; that object exposes dozens of attributes, most as plain numbers, a few like .ionenergies as lists since an atom has more than one ionisation energy.
from mendeleev import element # look up the element iron, using its periodic table symbol "Fe" Fe = element("Fe") # how strongly this element pulls on shared electrons; 1.83 for iron Fe.electronegativity # the typical size of one iron atom, in picometres: 126 pm Fe.atomic_radius # energy needed to remove one electron from an iron atom; an atom # can lose more than one electron, so this is a list, not one number Fe.ionenergies[1] # how iron's electrons are arranged around its nucleus, as text Fe.electron_configuration # iron's density: 7.874 grams per cubic centimetre Fe.density
Before the next slide throws real code at you, here is every symbol in it, explained once, assuming you have never written a line of code before. Every example below is taken directly from the Materials Project code on the next slide.
importPython starts nearly empty. If you want to use a toolbox someone else built, like the one for talking to the Materials Project, you have to say so first. Writing import is that request: "fetch this toolbox, and make everything inside it available to me from this point onward." Without it, Python has no idea the toolbox even exists.
from … import …A toolbox can hold dozens of tools you will never use. from mp_api.client import MPRester means: "reach inside the mp_api.client toolbox, and hand me only the one tool called MPRester. Leave everything else in the box." It is more precise, and faster, than importing everything.
.A dot always means the same thing: step inside. mp_api.client reads as "inside the mp_api toolbox, there is a smaller box called client." mpr.materials.summary.search repeats that idea three times in a row: inside mpr, find materials; inside materials, find summary; inside summary, find the tool called search.
asas mpr means "and from now on, call this thing mpr instead of typing its full name every time." Python does not care what nickname you pick, you could write as banana and it would still work. mpr is chosen because it reminds a human reader "this is my MP Rester connection."
( )Round brackets right after a name almost always mean "carry out this action, and here is what to do it with." MPRester("YOUR_API_KEY") means: run MPRester, and hand it your API key to work with. Whatever sits inside the brackets is called an argument, the input the action actually needs.
( , )Round brackets have a second, unrelated job: when they hold values separated by commas and are not attached to a name, like (2, 3), they build a small fixed group called a tuple. It is a strict, ordered pair: first the number 2, then the number 3, nothing can be added or removed later.
[ ]Square brackets build a list: items kept in the exact order you wrote them, and you can add more later. ["Fe", "O"] is a list holding exactly two items, the text "Fe" and the text "O", in that order.
{ }Curly braces build a dictionary, a set of labels each pointing to a value, exactly like looking up a word in a real dictionary to find its meaning. You will meet these from the next database onward, in the shape {"limit": 10}: the label "limit" points to the value 10.
" "Anything wrapped in quotation marks is a string: a literal piece of text, exactly as typed, not a number and not an instruction to run. "Fe" and "YOUR_API_KEY" are both strings. Without the quotes, Python would think you were referring to a variable named Fe, which does not exist.
=A single = does not mean "equals" the way it does in a maths class. It means "take whatever is on the right, and remember it under the name on the left." docs = mpr.materials.summary.search(...) means: run that search, then keep the answer, and call it docs from now on.
Both mpr and docs are variables: names you invented yourself to remember something by, exactly like writing a label on a box so you know what is inside without opening it every single time. Python places no meaning on the name itself, mpr could be called connection instead and work identically. Good names are chosen for the next human reader, not for the computer.
Inside mp_api.client: one class, MPRester, exposes many endpoint groups as attributes, mpr.materials.summary, .electronic_structure, .elasticity, .phonon, .thermo, each with its own .search(). A few common lookups, like fetching a structure by ID, get their own shortcut method directly on mpr.
Available properties:
num_elements=(2, 3), word by wordnum_elements is the name of one specific question the Materials Project lets you ask: "how many different elements is this material made of?" The = assigns your answer to that question. (2, 3) is a tuple, a fixed, two-part pair in round brackets (see the previous slide): the Materials Project reads a pair here as a range, minimum first, maximum second. So this line says "keep only materials made of at least 2 and at most 3 different elements." A material with exactly 2 elements is called binary (like Fe2O3: iron and oxygen); exactly 3 is called ternary. Writing 2 and 3 together, in one filter, keeps both kinds and excludes everything else: a single pure element (1) or anything with 4 or more.
# MPRester is the one class you need; it is the "front desk" # for every query you send to the Materials Project from mp_api.client import MPRester # "with" opens the connection here, and automatically closes it # again once you are done, even if something goes wrong partway through with MPRester("YOUR_API_KEY") as mpr: # "summary" is one category of data the Materials Project offers; # .search() is how you ask it a question docs = mpr.materials.summary.search( # only return materials that contain both iron and oxygen elements=["Fe", "O"], # only materials made of exactly 2 or exactly 3 different elements num_elements=(2, 3), # of everything the database knows about each material, only # send back these five specific pieces of information fields=[ "material_id", # its unique ID, like "mp-19770" "formula_pretty", # its chemical formula, like "Fe2O3" "band_gap", # how good an insulator it is, in eV "formation_energy_per_atom", # how stable it is, roughly "is_stable" # true or false: is it actually stable ] ) # "docs" now holds a list of matching materials, one entry each # a convenient shortcut for one common task: fetching the full # atom-by-atom structure of one specific material by its ID struct = mpr.get_structure_by_material_id( "mp-19770" # this ID happens to be hematite, Fe2O3 )
The backbone of computational materials analysis in Python. Handles crystal structures, phase diagrams, symmetry analysis, and file I/O.
What's inside: pymatgen.core holds the core objects (Structure, Lattice, Species, Composition); pymatgen.io has one submodule per file format (.cif, .vasp, .xyz); pymatgen.symmetry wraps the C library spglib for space groups; pymatgen.analysis covers phase diagrams, bonding, and structure matching.
.core?Same rule as always: a dot means "step one level deeper inside." pymatgen is the big toolbox; core is the name pymatgen's authors gave to one specific smaller box inside it, the one holding the foundational, load-bearing tools that almost everything else in pymatgen depends on, most importantly the Structure class itself. Other smaller boxes inside pymatgen have different names for different jobs: io for reading and writing files, symmetry for space groups, analysis for higher-level calculations. "core" specifically signals "the basics live here."
# Structure is the central object: it represents one crystal, # atoms, positions, and unit cell, all bundled together from pymatgen.core import Structure # open the file and build a Structure from it; pymatgen figures # out it is a CIF file just from the ".cif" ending, automatically struct = Structure.from_file("Fe2O3.cif") # once you have a Structure, its properties are ready instantly, # pymatgen already computed them when the file was read # the lengths of the unit cell's three edges, in Ångstroms print(struct.lattice.abc) # the three angles between those edges, in degrees print(struct.lattice.angles) # the chemical formula, printed as text: "Fe2 O3" print(struct.formula) # the density: how much mass is packed into the unit cell's volume print(struct.density) # the volume of the unit cell itself print(struct.volume) # this tool figures out a crystal's symmetry; behind the scenes # it hands the work off to spglib, a separate, well-trusted library from pymatgen.symmetry.analyzer import SpacegroupAnalyzer # run the symmetry analysis on our structure, once, and keep the result sga = SpacegroupAnalyzer(struct) # the symmetry group's short name, e.g. "R-3c" print(sga.get_space_group_symbol()) # the same symmetry group, but as a number from 1 to 230, # since there are exactly 230 possible space groups print(sga.get_space_group_number()) # rebuild the structure using the standard, textbook version of # its unit cell shape, useful for comparing structures fairly conv = sga.get_conventional_standard_structure()
Turns crystal structures and compositions into numerical feature vectors that machine learning models can consume. Includes built-in datasets.
What's inside: matminer.featurizers is organised by input type, .composition for formula-only featurizers like ElementProperty, .structure for full-structure ones, .site for per-atom descriptors, and .conversions for type converters like StrToComposition. matminer.datasets ships benchmark datasets ready to load in one call.
Arguments without a name. stc.featurize_dataframe(df, "structure") hands over two values with no name= in front of them. These are called positional arguments: Python matches them to what the tool expects purely by the order you list them in, first value to the first slot, second to the second. Compare that to col_id="composition" a few lines later, a keyword argument, where you say by name which slot it fills, so the order no longer matters.
Calling a tool on its own name. ElementProperty.from_preset("magpie") looks like the dot-and-parentheses pattern you already know, but notice there is no variable sitting before the dot yet, only the class name itself. This is a factory: instead of building an empty ElementProperty and configuring it yourself step by step, from_preset hands you back one ready-made, pre-tuned, exactly like ordering a preset meal instead of assembling one from raw ingredients.
# load_dataset fetches a ready-made benchmark dataset in one call from matminer.datasets import load_dataset # ElementProperty is the tool that runs MAGPIE, matminer's most # popular way of turning a formula into numbers from matminer.featurizers.composition import \ ElementProperty # this converts a formula written as plain text, like "Fe2O3", # into a proper Composition object matminer can work with from matminer.featurizers.conversions import \ StrToComposition # download (only the first time) and load the Matbench band-gap # dataset, over 106,000 materials, straight into a table df = load_dataset("matbench_mp_gap") # every featurizer needs a proper Composition object to work from, # not a plain text string, so convert the column first stc = StrToComposition() df = stc.featurize_dataframe(df, "structure") # "magpie" is a named, ready-tuned recipe: 22 elemental properties, # each summarised 6 different statistical ways ep = ElementProperty.from_preset("magpie") # run that recipe on the "composition" column; this is the step # that actually adds 145 new numeric columns to the table df = ep.featurize_dataframe( df, col_id="composition" ) # check the table's size: same number of rows, many more columns print(df.shape)
AFLOW (Automatic Flow) is the largest high-throughput DFT database. Access via the aflow Python package or directly via REST.
What's inside the aflow package: aflow.search() starts a query; aflow.K is a namespace listing every AFLOW property as an attribute, so aflow.K.Egap is the band gap keyword, not a string you have to remember correctly; .filter(), .select(), and .orderby() chain onto the query object.
Method chaining. .filter(...).filter(...).select(...) stacks several dots-and-parentheses in a row. Each call finishes and hands back an updated version of the same query, which the next dot immediately acts on again, like giving a series of instructions to the same assistant one after another: "narrow this, narrow it again, now pick these columns."
Comparison symbols. aflow.K.Egap > 0 is a question, not an instruction: "is this value greater than 0?" It resolves to True or False. < means less than, and == (two equals signs, not one) means "is equal to." Two equals signs are used here deliberately, because a single = already means something else: storing a value, not asking a question.
The for … in loop. for entry in results[:20]: means "one at a time, take each item out of results, temporarily call it entry, and run the indented lines below using it." It repeats until every item has had its turn.
Slicing with a colon. results[:20] means "take items from the start, up to but not including position 20," a slice of the full list rather than the whole thing. Leaving the number before the colon blank means "start from the very beginning."
.filter() calls to narrow results by any property.pip install aflow
import aflow # "icsd" restricts the search to materials that were first found # experimentally, and later re-computed with DFT for consistency results = ( aflow.search(catalog="icsd") # only keep materials with a band gap above zero, which # rules out metals (a metal's band gap is exactly 0) .filter(aflow.K.Egap > 0) # and below 3.5 eV, which rules out very wide-gap insulators, # keeping us in the semiconductor range .filter(aflow.K.Egap < 3.5) # only keep materials built from exactly two elements .filter(aflow.K.nspecies == 2) # of everything AFLOW knows, only send back these four fields .select( aflow.K.compound, # the formula, e.g. "GaAs" aflow.K.Egap, # the band gap, in eV aflow.K.spacegroup_relax, # its symmetry group aflow.K.energy_atom # energy per atom, a stability clue ) # sort the results from smallest band gap to largest .orderby(aflow.K.Egap) ) # nothing has actually been sent to AFLOW's servers yet: this whole # query is just a description of what you want, built step by step # the request only fires once you actually try to use the results, # which is what this loop does; here we look at the first 20 for entry in results[:20]: # each result lets you read its fields by name, like this print(entry.compound, entry.Egap) # if the aflow package cannot do what you need, you can always # fall back to AFLOW's plain REST API directly, using requests import requests url = "http://aflow.org/API/aflowlib.php" r = requests.get(url, params={ "species": "Fe,O", # the material must contain these elements "format": "json" # ask for a computer-readable answer, not a webpage })
Open Quantum Materials Database, 1 million DFT-computed formation enthalpies and stability data. No Python client needed: pure REST.
What's inside the response: every OQMD API call returns a JSON object with a "data" key holding a list of records, and separate "meta" info about pagination. Each record is a flat dictionary, one key per requested field, which is exactly the shape pd.DataFrame() expects.
f before a stringYou have seen strings in quotation marks before, but f"{BASE}/formationenthalpy" has an f sitting right before the opening quote. That letter turns on a feature called an f-string: anything inside curly braces gets swapped out for the current value of that variable before the string is used. If BASE holds the text "https://oqmd.org/oqmdapi", then f"{BASE}/formationenthalpy" becomes "https://oqmd.org/oqmdapi/formationenthalpy", built fresh, without you having to glue the pieces together by hand. Also new here: <= means "less than or equal to," the same idea as > and < from the AFLOW slide, just inclusive of the boundary itself.
https://oqmd.org/oqmdapi/import requests import pandas as pd # every question we ask OQMD starts with this same web address BASE = "https://oqmd.org/oqmdapi" # "/formationenthalpy" is one specific section of OQMD's API, # focused on how stable a material is r = requests.get( f"{BASE}/formationenthalpy", # these extra options narrow down exactly what comes back params={ "composition": "Fe-O", # any material made of iron and oxygen "limit": 100, # send back at most 100 results "offset": 0, # start counting from the very first result "fields": "name,delta_e,stability,band_gap" # only these 4 pieces of info } ) # turn the raw answer into a Python dictionary we can work with data = r.json() # the actual list of materials lives inside data, under "data" df = pd.DataFrame(data["data"]) # a quick look at the first 5 rows, just to sanity-check the data print(df.head()) # "stability" measures how far above the most stable possible # combination a material sits; zero or below means it is stable stable = df[df["stability"] <= 0.0] # print how many materials passed that stability check print(f"Stable phases: {len(stable)}")
The Crystallography Open Database contains experimental structures from X-ray and neutron diffraction, not DFT. Use when you need real measured structures.
What's inside this code: pymatgen.io.cif.CifParser reads raw CIF text and builds one or more Structure objects, since a CIF file can describe several structures at once; Python's built-in io.StringIO wraps a text string so it can be read like a file, without ever writing anything to disk.
Brackets stacked back to back. entries[0]["file"] reads right to left, one step at a time. entries[0] means "take the first item out of the entries list" (list positions start counting from 0, not 1, so [0] is the very first one). That first item turns out to itself be a dictionary, so the second bracket, ["file"], immediately looks up the value stored under the label "file" inside it.
A property with no parentheses. requests.get(cif_url).text ends in .text with no round brackets after it. That absence is meaningful: parentheses mean "run an action," so leaving them off means this is not an action to run, it is a value already sitting there waiting to be read, in this case the response's raw text content.
import requests # CifParser reads raw CIF text and turns it into one or more # pymatgen Structure objects from pymatgen.io.cif import CifParser # StringIO is a small trick: it lets Python treat a plain piece # of text as if it were an open file, without saving anything to disk from io import StringIO COD = "https://www.crystallography.net/cod" # Step 1: search for materials with this formula; this only # gives you a list of matches, not the actual structures yet r = requests.get( f"{COD}/result", params={"formula": "TiO2", "format": "json"} ) entries = r.json() # COD often stores several different measurements of the same # material, so a search like this can return many entries print(f"Found {len(entries)} entries") # Step 2: each entry has its own ID, and the actual CIF file for # it always lives at the same kind of web address cod_id = entries[0]["file"] cif_url = f"{COD}/{cod_id}.cif" # download that CIF file's contents as plain text cif_text = requests.get(cif_url).text # Step 3: hand that text straight to pymatgen, using StringIO, # so there is no need to save a temporary file first parser = CifParser(StringIO(cif_text)) # a single CIF file can describe more than one structure, # so this returns a list; here we just take the first one struct = parser.get_structures()[0] # from here on, "struct" is a normal Structure object, exactly # like the one we got from the Materials Project earlier print(struct.formula, struct.density)
NOMAD stores the raw output files of DFT calculations, not just extracted properties. Useful for accessing calculation metadata and diverse code outputs.
What's inside the query: the "query" block filters which entries match, "pagination" controls how many come back per request, and "required" tells NOMAD which sections of each entry to actually send, a "*" means all fields in that section, without pulling the full raw calculation files.
.get()You already know ["key"] looks something up inside a dictionary, but if that key does not exist, Python stops the whole program with an error. mat.get("symmetry", {}) is a gentler version of the same idea: "look up symmetry, and if it is not there, quietly hand me an empty dictionary instead of crashing." The .get("space_group_number") chained straight after it then does the same safe lookup one level deeper. This matters here because not every NOMAD entry records every field, so the code needs to keep going even when something is missing.
Base URL:
https://nomad-lab.eu/prod/v1/api/v1/
import requests URL = "https://nomad-lab.eu/prod/v1/api/v1" # unlike the other databases on this slide, NOMAD wants its # question written out as a structured dictionary, sent as POST payload = { # the actual filter: which materials qualify "query": { "results.material.elements": { "all": ["Fe", "Ni"] # must contain both iron and nickel } }, # how many results to send back at once, like a page size "pagination": {"page_size": 20}, # of everything NOMAD could tell you about each match, only # send back these two categories, not the full raw calculation "required": { "results": { "material": "*", # everything about the material itself "method": "*" # everything about how it was calculated } } } # "post" (rather than "get") is used because the question is too # complex to fit neatly into a web address; it travels as the payload r = requests.post( f"{URL}/entries/query", json=payload # requests turns this dictionary into JSON automatically ) # pull the list of matching entries out of the response hits = r.json()["data"] for h in hits: # the material's details sit a few levels deep in the response mat = h["results"]["material"] # print its formula, and its space group number if one was # recorded; .get() quietly returns nothing instead of crashing # when a field happens to be missing print(mat["chemical_formula_reduced"], mat.get("symmetry", {}).get("space_group_number"))
A machine learning model cannot read a crystal structure – it only understands lists of numbers. Featurization is the process of converting a crystal into those numbers. Each number is called a feature.
You can't feed a house photo into a price-prediction model. But you can describe the house with numbers: rooms, floor area (m²), distance to school (km), age (years). Each of these is a feature. Crystal featurization works the same way: instead of a house, you describe a crystal.
Looks at the chemical formula. For each element (e.g. Fe, O in Fe₂O₃, looks up 22 elemental properties – electronegativity, atomic radius, valence electrons, melting point … Then computes statistics (mean, range, std) across elements. Result: 145 numbers from just the formula.
matminer in action
from matminer.datasets import load_dataset from matminer.featurizers.composition import ElementProperty from matminer.featurizers.conversions import StrToComposition # load a ready-made set of 106,113 materials, each already # labelled with its real, DFT-computed band gap df = load_dataset("matbench_mp_gap") # turn the formula text, like "Fe2O3", into a proper Composition # object, which is the input every featurizer expects stc = StrToComposition() df = stc.featurize_dataframe(df, "structure") # MAGPIE is the featurizer: it turns each composition into 145 # separate numbers describing that material ep = ElementProperty.from_preset("magpie") df = ep.featurize_dataframe(df, col_id="composition") # the table now has 147+ columns instead of 2: it is ready to # feed straight into a machine learning model print(df.shape)
Think of building a house – you need different tradespeople for different jobs. These four libraries are your four tradespeople for data analysis.
Gives you a table (DataFrame), like a very powerful Excel, but controlled by Python. Each row is one material, each column is one property or feature. Filter rows, sort, merge tables with .merge(), group and summarise with .groupby(). Every database result goes into a pandas DataFrame first.
Does mathematical operations on thousands of values at once in a single line. Divide 100,000 band gaps by 2, one instruction, via numpy.linalg for matrix algebra and numpy.random for sampling. The scientific calculator that works on entire tables simultaneously. pandas, scikit-learn, and matminer are all built on numpy internally.
Contains dozens of ML algorithms across sklearn.ensemble (Random Forest), sklearn.linear_model (Linear Regression, ElasticNet), and sklearn.svm. Give it your feature table and target property, it trains a model that learns the relationship. Once trained, it predicts properties of materials it has never seen before.
Draws graphs and charts from your results, predicted vs actual band gaps, error histograms, learning curves, through the pyplot interface, itself built on Figure and Axes objects for multi-panel layouts. Takes numbers and turns them into pictures you can understand and share. Every result in a paper started as a matplotlib figure.
Same four libraries as the rest of this lecture, now chained: mp_api.client fetches, matminer.featurizers.conversions and .composition featurize, sklearn.ensemble and .model_selection train and score.
[{"structure": d.structure, "band_gap": d.band_gap} for d in docs] looks dense, but it is the for … in loop you already met on the AFLOW slide, just written inside square brackets instead of spread across several indented lines. Read it as: "for every d in docs, build a small dictionary out of it, and collect all of those dictionaries into one list." This is called a list comprehension, a compact way to write "loop over this, and build a new list from what you find," when the whole loop only does one simple thing per item.
Formatting a number inside an f-string. f"R² = {scores.mean():.3f}" adds one more trick to the f-strings from the OQMD slide: the :.3f right after the value is a formatting instruction meaning "show this as a decimal number, rounded to exactly 3 digits after the point," so 0.8724193 becomes the much more readable 0.872.
from mp_api.client import MPRester # step 1 from matminer.featurizers.composition import ElementProperty # step 3 from matminer.featurizers.conversions import StructureToComposition # step 2 import pandas as pd # carries the data through every step from sklearn.ensemble import RandomForestRegressor # step 4 from sklearn.model_selection import cross_val_score # step 4 # STEP 1, FETCH: ask the Materials Project for semiconductors, # meaning a band gap somewhere between 0.1 and 5.0 eV: this range # excludes metals on one side and very wide-gap insulators on the other with MPRester("YOUR_API_KEY") as mpr: docs = mpr.materials.summary.search( fields=["structure", "band_gap"], band_gap=(0.1, 5.0) ) # "docs" is now a list, one entry per matching material # build a table with one row per material, keeping just the two # things we actually need: its structure and its band gap df = pd.DataFrame([{ "structure": d.structure, "band_gap": d.band_gap # this is the value we want the model to learn to predict } for d in docs]) # STEP 2, PARSE: pull the chemical composition, e.g. "Fe2O3", out # of each full structure, since MAGPIE only needs the formula stc = StructureToComposition() df = stc.featurize_dataframe(df, "structure") # STEP 3, FEATURIZE: turn each composition into 145 numbers using # MAGPIE, the same preset explained back on slide 13 ep = ElementProperty.from_preset("magpie") df = ep.featurize_dataframe(df, "composition") # STEP 4, TRAIN: separate the 145 feature columns (the inputs) # from the band_gap column (what we are trying to predict) X = df[ep.feature_labels()] y = df["band_gap"] # build a Random Forest model made of 100 individual decision # trees; at this point it has not learned anything yet rf = RandomForestRegressor(n_estimators=100) # train and test the model 5 separate times, on 5 different # slices of the data, so one lucky or unlucky split cannot fool us scores = cross_val_score(rf, X, y, scoring="r2", cv=5) # print the average score across all 5 tests: closer to 1.0 means # the model's predictions closely match the real band gaps print(f"R² = {scores.mean():.3f}")
MCP is an open standard that lets AI models connect to external data sources and tools through a single, unified interface. Think of it as USB-C for AI: instead of writing a custom Python script for every database, you connect once and any AI agent can query any MCP-compatible source in plain language.
Announced by Anthropic in November 2024 and released as open-source. Now maintained as a community standard with SDKs in Python, TypeScript, Java, Kotlin, C#, and Swift.
Before MCP, every AI tool had its own custom connector for every data source: one integration for Slack, another for GitHub, another for a database. Each one was written from scratch. The result was an M × N problem: M tools each needing N connectors. MCP collapses this to M + N: each tool implements MCP once, each data source exposes an MCP server once, and everything works together.
| Library / Tool | Purpose | Access method | Best for |
|---|---|---|---|
| mp-api | Materials Project client | Python client | DFT band gaps, formation energies, structures |
| aflow | AFLOW database client | Python client | Alloys, largest compound space, electronic structure |
| requests | Generic HTTP client | REST API | OQMD, COD, NOMAD – any database with an API |
| pymatgen | Structure analysis | Library | Parsing CIF/POSCAR, symmetry, phase diagrams |
| matminer | Feature extraction | Library | Turning structures into ML-ready feature vectors |
| ASE | Atomic Simulation Env. | Library | MD/DFT setup, file conversion, visualisation |
Hands-on: querying the Materials Project, building a band gap prediction dataset, and training your first ElasticNet model.