Skip to content

Welcome to blackbar-py

blackbar-py available at https://github.com/bnosac/blackbar-py provides a python package called blackbar which providing the backend facilities to allow to Anonymize and Pseudonimize texts. It has the following functionalities

Connect to the blackbar database

  • BlackbarDB class and the methods to interact with it
    • read/sendquery/read_documents/read_patientdoctoraddresses
    • update_anonimisation/read_blackbar_status/read_anonimisation + parsing the anonimization json: parse_anonimisation_json
  • Deprecated database connectivity (which are covered in class BlackbarDB)
    • read_iris, read_iris_sample, read_iris_documents, read_iris_patientartsadressen, iris_update_cvt, iris_sendquery

Store models and other files on S3 in particular using Minio

  • blackbar_s3_list, blackbar_s3_upload, blackbar_s3_download (not model specific)
  • blackbar_model_save, blackbar_model_load (model specific)

Entry point to anonymise and pseudonymize texts + structure the output

  • class Blackbar and the methods anonimise, anonimise_extended
  • deid_anonimise_dataframe
  • Pseudonymization
    • PseudoGenerator (generate pseudo text)
    • anonimisation_entities (extract the anonimized entities)
    • pseudo_replacements, txt_mimic_readability (replacement functions while sticking as close to the original layout as possible)
    • Utility functions which help with the pseudonymization
      • txt_insert, txt_freq, txt_leading_trailing
      • txt_n_capital, txt_contains, txt_contains_lot_of_capitals, txt_n_newlines

Interact with Inception and the Inception API

  • Connectivity: inception_client, inception_list_projects,
  • Exporting and fetching annotations: inception_export, inception_list_documents, inception_list_annotations
  • Inspecting the annotations with their corresponding types: inception_types, inception_cas, read_xmi
  • Fetching the log of the user interactions on the site: inception_read_eventlog
  • Utiltiy functions to extract information from the inception data:
    • blackbar_inception_entities, line_spans, token_spans, token_entity_spans

Deploy docker containers

Functions to inspect, build, maintain and deploy containers using Docker and Podman with functions

  • pod_info, pod_ls, pod_pull, pod_build,
  • pod_container, pod_container_log
  • pod_remove

Custom functionalities for specific hospitals

  • uzb_identify_chunks, uzb_harmonize_physician, uzb_vn_achternaam, uzb_txt_contains

General utility functions

  • ascii_translit, txt_clean_word2vec
  • txt_sample, txt_paste
  • na_exclude
  • chunk
  • tokenize_letters, tokenize_spaces_punct, tokenize_lines
  • combine_chunkranges to allow to combine ranges of entities from different models