Feedback

Under construction.

You can import the results of the anonymization and pseudonymization flow back in Inception for automatic creation of training data.

Note

This is especially usefull at the start of the project where you are evaluating the models and as well in order to use the detection by the Smith-Waterman algorithm to improve the NLP models. In this way the name/address detection can be improved.

First of all load the results of the anonymization and pseudonymization flows which are stored in the blackbar_document and blackbar_deid tables by using function ´blackbar_annotations´ functionality by specifying a set of documents which you would like to have shown in the frontend.

from blackbar import BlackbarDB
from blackbar import blackbar_annotations, blackbar_cas
db = BlackbarDB("IRIS")
db = BlackbarDB("test")
##
## Make sure you have run the pseudonymization such that the results are in your database
##
#info = blackbar_s3_download(name = "deid_v2", bucket = "blackbar-models")
#deid = Blackbar(info)
#pseudo  = PseudoGenerator(locale = "nl_BE")
#docs = db.read_documents(ids = [1, 2, 3], type = "deid")
#anno = deid_anonymize(deid, docs, type = "_", extended = True)
#anno = db.read_anonymization(ids = anno["doc_id"])
#results = deid_pseudonymize(anno, pseudo = pseudo, dateshift = {"years": -3, "months": -3, "days": -167, "weeks": 4, "days": 23}, failure_strategy = None)
#db.update(results, type = "pseudonymization", project_id = 54321, status = 2)

anno = blackbar_annotations(db, sql = "select * from blackbar_document")
anno = blackbar_annotations(db, ids = list(range(10)))
x = blackbar_cas(anno)
# inception_upload_documents(x, project_id = 46)

Named Entity Recognition models
Smith-Waterman local alignment

Text similarity metrics

  >>> from blackbar import blackbar_annotations, BlackbarDB
  >>> db = BlackbarDB("IRIS")
  >>> db = BlackbarDB("test")
  >>> anno = blackbar_annotations(db, sql = "select * from blackbar_document")
  >>> anno = blackbar_annotations(db, ids = list(range(10)))