ftm aggregate
will retain the entire dataset in memory, which is impossible to do for large databases. In such cases, it's recommended to use an on-disk entity aggregation tool, followthemoney-store
.ftm map
is that the source data is actually referenced within the YAML mapping file as an absolute URL. While this makes sense for data sourced from a SQL database or a public CSV file, you might sometimes want to map a local CSV file instead. For this, a modified version of ftm map
is provided, ftm map-csv
. It ignores the specified source URLs and reads data from standard input:People
go into one, Directorships
into another.ftm export-excel
command:-o
option.ftm export-csv
, the exporter will usually generate multiple output files, one for each schema of entities present in the input stream of Follow the Money entities. To handle this, it expects to be given a directory name:Person.csv
, LegalEntity.csv
, Vessel.csv
, etc.Directorship
, Ownership
, Family
or Payment
, contain annotations that define how they can be transformed into an edge with a source and target.Email
has an emitters
property that refers to a LegalEntity
, the sender. The emitters
property connects the two entities and can also be turned into an edge.email
, iban
, names
) can be formed into nodes, with edges formed towards each node that derives from an entity with that property value. For example, an address
node for "40 Wall Street" would show links to all the companies registered there, or a node representing the name "Frank Smith" would connect all the documents mentioning that name.cypher-shell
command to load the US sanctions list into a local instance of the database:-e
option:neo4-admin import
command to load them into an empty database. This requires shutting down the Neo4J server and then running a command that will write the new database.neo4-admin
import command that should be used to load the data into a graph store.-e
flag for property types that should be turned into nodes:NTriples
format.--unqualified
flag:ftm
includes a function to transform a stream of OCDS objects into FtMs Contracts
and ContractAwards
. This was developed in particular to import data from the DIGIWHIST OpenTender.eu site, so other implementations of OCDS may require extending the importer to accommodate other formats.followthemoney-store
instead of ftm aggregate
.ftm aggregate
command.followthemoney-store
is available as a Python library and a command line tool. It can use any SQL database as a backend, with a local SQLite file set as a default. When using PostgreSQL as a database, followthemoney-store
can use its built-in upsert functionality, making the backend more performant than others.followthemoney-store
with SQLite, install it like this:followthemoney-store
command in read or write mode: