ftm aggregatewill retain the entire dataset in memory, which is impossible to do for large databases. In such cases, it's recommended to use an on-disk entity aggregation tool,
ftm mapis that the source data is actually referenced within the YAML mapping file as an absolute URL. While this makes sense for data sourced from a SQL database or a public CSV file, you might sometimes want to map a local CSV file instead. For this, a modified version of
ftm mapis provided,
ftm map-csv. It ignores the specified source URLs and reads data from standard input:
Peoplego into one,
ftm export-csv, the exporter will usually generate multiple output files, one for each schema of entities present in the input stream of Follow the Money entities. To handle this, it expects to be given a directory name:
Payment, contain annotations that define how they can be transformed into an edge with a source and target.
emittersproperty that refers to a
LegalEntity, the sender. The
emittersproperty connects the two entities and can also be turned into an edge.
names) can be formed into nodes, with edges formed towards each node that derives from an entity with that property value. For example, an
addressnode for "40 Wall Street" would show links to all the companies registered there, or a node representing the name "Frank Smith" would connect all the documents mentioning that name.
cypher-shellcommand to load the US sanctions list into a local instance of the database:
neo4-admin importcommand to load them into an empty database. This requires shutting down the Neo4J server and then running a command that will write the new database.
neo4-adminimport command that should be used to load the data into a graph store.
-eflag for property types that should be turned into nodes:
ftmincludes a function to transform a stream of OCDS objects into FtMs
ContractAwards. This was developed in particular to import data from the DIGIWHIST OpenTender.eu site, so other implementations of OCDS may require extending the importer to accommodate other formats.
followthemoney-storeis available as a Python library and a command line tool. It can use any SQL database as a backend, with a local SQLite file set as a default. When using PostgreSQL as a database,
followthemoney-storecan use its built-in upsert functionality, making the backend more performant than others.
followthemoney-storewith SQLite, install it like this:
followthemoney-storecommand in read or write mode: