NLnet foundation has supported the PyCM project from version 3.6 to 4.0 through the NGI Assure Fund. This fund is set up by NLnet foundation with funding from the European Commission's Next Generation Internet program, administered by DG Communications Networks, Content, and Technology under grant agreement No 957073.

Python Software Foundation (PSF) grants PyCM library partially for version 3.7. PSF is the organization behind Python. Their mission is to promote, protect, and advance the Python programming language and to support and facilitate the growth of a diverse and international community of Python programmers.

 Papers and Reports

 Version 4.0

  • pycmMultiLabelError class added
  • MultiLabelCM class added
  • get_cm_by_class method added
  • get_cm_by_sample method added
  • __mlcm_vector_handler__ function added
  • __mlcm_assign_classes__ function added
  • __mlcm_vectors_filter__ function added
  • __set_to_multihot__ function added
  • deprecated function added
  • Document modified
  • README.md modified
  • Example-4 modified
  • Test system modified
  • Python 3.5 support dropped

 Source Code (.Zip) (943 KB)

 Source Code (.tar.gz) (908 KB)

 Version 3.9

  • OVERALL_PARAMS dictionary added
  • __imbalancement_handler__ function added
  • vector_serializer function added
  • NPV micro/macro added
  • log_loss method added
  • 23 new distance/similarity added
    1. Dennis
    2. Digby
    3. Dispersion
    4. Doolittle
    5. Eyraud
    6. Fager & McGowan
    7. Faith
    8. Fleiss-Levin-Paik
    9. Forbes I
    10. Forbes II
    11. Fossum
    12. Gilbert & Wells
    13. Goodall
    14. Goodman & Kruskal's Lambda
    15. Goodman & Kruskal Lambda-r
    16. Guttman's Lambda A
    17. Guttman's Lambda B
    18. Hamann
    19. Harris & Lahey
    20. Hawkins & Dotson
    21. Kendall's Tau
    22. Kent & Foster I
    23. Kent & Foster II
  • metrics_off parameter added to ConfusionMatrix __init__ method
  • CLASS_PARAMS changed to a dictionary
  • Code style modified
  • sort parameter added to relabel method
  • Document modified
  • CONTRIBUTING.md updated
  • codecov removed from dev-requirements.txt
  • Test system modified

 Source Code (.Zip) (940 KB)

 Source Code (.tar.gz) (906 KB)

 Version 3.8

  • distance method added
  • __contains__ method added
  • __getitem__ method added
  • Goodman-Kruskal's Lambda A benchmark added
  • Goodman-Kruskal's Lambda B benchmark added
  • Krippendorff's Alpha benchmark added
  • Pearson's C benchmark added
  • 30 new distance/similarity added
    1. AMPLE
    2. Anderberg's D
    3. Andres & Marzo's Delta
    4. Baroni-Urbani & Buser I
    5. Baroni-Urbani & Buser II
    6. Batagelj & Bren
    7. Baulieu I
    8. Baulieu II
    9. Baulieu III
    10. Baulieu IV
    11. Baulieu V
    12. Baulieu VI
    13. Baulieu VII
    14. Baulieu VIII
    15. Baulieu IX
    16. Baulieu X
    17. Baulieu XI
    18. Baulieu XII
    19. Baulieu XIII
    20. Baulieu XIV
    21. Baulieu XV
    22. Benini I
    23. Benini II
    24. Canberra
    25. Clement
    26. Consonni & Todeschini I
    27. Consonni & Todeschini II
    28. Consonni & Todeschini III
    29. Consonni & Todeschini IV
    30. Consonni & Todeschini V
  • relabel method sort bug fixed
  • README.md modified
  • Compare overall benchmarks default weights updated
  • Document modified
  • Test system modified

 Source Code (.Zip) (927 KB)

 Source Code (.tar.gz) (892 KB)

 Version 3.7

  • Curve class added
  • ROCCurve class added
  • PRCurve class added
  • pycmCurveError class added
  • CONTRIBUTING.md updated
  • matrix_params_calc function optimized
  • README.md modified
  • Document modified
  • Test system modified
  • Python 3.11 added to test.yml

 Source Code (.Zip) (916 KB)

 Source Code (.tar.gz) (881 KB)

 Version 3.6

  • Hamming distance added
  • Braun-Blanquet similarity added
  • classes parameter added to matrix_params_from_table function
  • Matrices with numpy.integer elements are now accepted
  • Arrays added to matrix parameter accepting formats
  • Website changed to http://www.pycm.io
  • Document modified
  • README.md modified

 Source Code (.Zip) (883 KB)

 Source Code (.tar.gz) (850 KB)

 Version 3.5

  • Anaconda workflow added
  • Custom iterating setting added
  • Custom casting setting added
  • plot method updated
  • class_statistics function modified
  • overall_statistics function modified
  • BCD_calc function modified
  • CONTRIBUTING.md updated
  • CODE_OF_CONDUCT.md updated
  • Document modified

 Source Code (.Zip) (871 KB)

 Source Code (.tar.gz) (836 KB)

 Version 3.4

  • Colab badge added
  • Discord badge added
  • brier_score method added
  • J (Jaccard index) section in Document.ipynb updated
  • save_obj method updated
  • Python 3.10 added to test.yml
  • Example-3 updated
  • Docstrings of the functions updated
  • CONTRIBUTING.md updated

 Source Code (.Zip) (870 KB)

 Source Code (.tar.gz) (834 KB)

 Version 3.3

  • __compare_weight_handler__ function added
  • is_imbalanced parameter added to ConfusionMatrix __init__ method
  • class_benchmark_weight and overall_benchmark_weight parameters added to Compare __init__ method
  • statistic_recommend function modified
  • Compare weight parameter renamed to class_weight
  • Document modified
  • License updated
  • AUTHORS.md updated
  • README.md modified
  • Block diagrams updated

 Source Code (.Zip) (865 KB)

 Source Code (.tar.gz) (830 KB)

 Version 3.2

  • classes_filter function added
  • classes parameter added to matrix_params_calc function
  • classes parameter added to __obj_vector_handler__ function
  • classes parameter added to ConfusionMatrix __init__ method
  • name parameter removed from html_init function
  • shortener parameter added to html_table function
  • shortener parameter added to save_html method
  • Document modified
  • HTML report modified

 Source Code (.Zip) (1.1 MB)

 Source Code (.tar.gz) (1.1 MB)

 Version 3.1

  • requirements-splitter.py added
  • sensitivity_index method added
  • Test system modified
  • overall_statistics function modified
  • HTML report modified
  • Document modified
  • References format updated
  • CONTRIBUTING.md updated

 Source Code (.Zip) (1.1 MB)

 Source Code (.tar.gz) (1.1 MB)

 Version 3.0

  • plot_test.py added
  • axes_gen function added
  • add_number_label function added
  • plot method added
  • combine method added
  • matrix_combine function added
  • Document modified
  • README.md modified
  • Example-2 deprecated
  • Example-7 deprecated
  • Error messages modified

 Source Code (.Zip) (1.1 MB)

 Source Code (.tar.gz) (1.1 MB)

 Version 2.9

  • notebook_check.py added
  • to_array method added
  • __copy__ method added
  • copy method added
  • average method refactored

 Source Code (.Zip) (1 MB)

 Source Code (.tar.gz) (1 MB)

 Version 2.8

  • label_map attribute added
  • positions attribute added
  • position method added
  • Krippendorff's Alpha added
  • Aickin's Alpha added
  • weighted_alpha method added
  • Single class bug fixed
  • CLASS_NUMBER_ERROR error type changed to pycmMatrixError
  • relabel method bug fixed
  • Document modified
  • README.md modified

 Source Code (.Zip) (1 MB)

 Source Code (.tar.gz) (1 MB)

 Version 2.7

  • average method added
  • weighted_average method added
  • weighted_kappa method added
  • pycmAverageError class added
  • Bangdiwala's B added
  • MATLAB examples added
  • Github action added
  • Document modified
  • README.md modified
  • relabel method bug fixed
  • sparse_table_print function bug fixed
  • matrix_check function bug fixed
  • Minor bug in Compare class fixed
  • Class names mismatch bug fixed

 Source Code (.Zip) (1 MB)

 Source Code (.tar.gz) (1 MB)

 Version 2.6

  • custom_rounder function added
  • complement function added
  • sparse_matrix attribute added
  • sparse_normalized_matrix attribute added
  • Net benefit (NB) added
  • Yule's Q interpretation (QI) added
  • Adjusted Rand index (ARI) added
  • TNR micro/macro added
  • FPR micro/macro added
  • FNR micro/macro added
  • sparse parameter added
  • header parameter added to save_csv method
  • Handler functions moved to pycm_handler.py
  • Error objects moved to pycm_error.py
  • Verified tests references updated
  • Verified tests moved to verified_test.py
  • Test system modified
  • CONTRIBUTING.md updated
  • Namespace optimized
  • README.md modified
  • Document modified
  • print_normalized_matrix method modified
  • normalized_table_calc function modified
  • setup.py modified
  • summary mode updated
  • Dockerfile updated
  • Python 3.8 added to .travis.yaml and appveyor.yml

 Source Code (.Zip) (1 MB)

 Source Code (.tar.gz) (1 MB)

 Version 2.5

  • __version__ variable added
  • Individual classification success index (ICSI) added
  • Classification success index (CSI) added
  • Example-8 (Confidence interval) added
  • install.sh added
  • autopep8.sh added
  • Dockerfile added
  • CI method added to
    • ACC
    • AUC
    • Overall ACC
    • Kappa
    • TPR
    • TNR
    • PPV
    • NPV
    • PLR
    • NLR
    • PRE
  • test.sh moved to .travis folder
  • Python 3.4 support dropped
  • Python 2.7 support dropped
  • AUTHORS.md updated
  • save_stat,save_csv and save_html methods Non-ASCII character bug fixed
  • Mixed type input vectors bug fixed
  • CONTRIBUTING.md updated
  • Example-3 updated
  • README.md modified
  • Document modified
  • CI attribute renamed to CI95
  • kappa_se_calc function renamed to kappa_SE_calc
  • se_calc function modified and renamed to SE_calc
  • CI/SE functions moved to pycm_ci.py
  • Minor bug in save_html method fixed

 Source Code (.Zip) (1 MB)

 Source Code (.tar.gz) (1 MB)

 Version 2.4

  • Tversky index (TI) added
  • Area under the PR curve (AUPR) added
  • FUNDING.yml added
  • AUC_calc function modified
  • Document modified
  • summary parameter added to save and stat methods
  • sample_weight bug in numpy array format fixed
  • Inputs manipulation bug fixed
  • Test system modified
  • Warning system modified
  • alt_link parameter added to save_html method and online_help function
  • Compare class tests moved to compare_test.py
  • Warning tests moved to warning_test.py

 Source Code (.Zip) (894 KB)

 Source Code (.tar.gz) (869 KB)

 Version 2.3

  • Adjusted F-score (AGF) added
  • Overlap coefficient (OC) added
  • Otsuka-Ochiai coefficient (OOC) added
  • save_stat and save_vector parameters added to save_obj method
  • Document modified
  • README.md modified
  • Parameters recommendation for imbalance dataset modified
  • Minor bug in Compare class fixed
  • pycm_help function modified
  • Benchmarks color modified

 Source Code (.Zip) (882 KB)

 Source Code (.tar.gz) (861 KB)

 Version 2.2

  • Negative likelihood ratio interpretation (NLRI) added
  • Cramer's benchmark (SOA5) added
  • Matthews correlation coefficient interpretation (MCCI) added
  • Matthews's benchmark (SOA6) added
  • F1 macro added
  • F1 micro added
  • Accuracy macro added
  • Compare class score calculation modified
  • Parameters recommendation for multi-class dataset modified
  • Parameters recommendation for imbalance dataset modified
  • README.md modified
  • Document modified
  • Logo updated

 Source Code (.Zip) (871 KB)

 Source Code (.tar.gz) (849 KB)

 Version 2.1

  • Adjusted geometric mean (AGM) added
  • Yule's Q (Q) added
  • Compare class and parameters recommendation system block diagrams added
  • Document links bug fixed
  • Document modified

 Source Code (.Zip) (790 KB)

 Source Code (.tar.gz) (768 KB)

 Version 2.0

  • G-Mean (GM) added
  • Index of balanced accuracy (IBA) added
  • Optimized precision (OP) added
  • Pearson's C (C) added
  • Compare class added
  • Parameters recommendation warning added
  • ConfusionMatrix equal method added
  • Document modified
  • stat_print function bug fixed
  • table_print function bug fixed
  • Beta parameter renamed to beta
  • Parameters recommendation for imbalance dataset modified
  • normalize parameter added to save_html method
  • pycm_func.py splitted
  • filter and check functions moved to pycm_util.py
  • RACC_calc and RACCU_calc functions exception handler modified
  • Docstrings modified

 Source Code (.Zip) (731 KB)

 Source Code (.tar.gz) (709 KB)

 Version 1.9

  • Automatic/Manual (AM) added
  • Bray-Curtis dissimilarity (BCD) added
  • CODE_OF_CONDUCT.md added
  • ISSUE_TEMPLATE.md added
  • PULL_REQUEST_TEMPLATE.md added
  • CONTRIBUTING.md added
  • X11 color names support for save_html method added
  • Parameters recommendation system added
  • Warning message for high dimension matrix print added
  • Interactive notebooks section (binder) added
  • save_matrix and normalize arguments added to save_csv method
  • README.md modified
  • Document modified
  • __init__ optimized
  • Document and examples output files moved to different folders
  • Test system modified
  • relabel method bug fixed

 Source Code (.Zip) (710 KB)

 Source Code (.tar.gz) (689 KB)

 Version 1.8

  • Lift Score (LS) added
  • color argument added to save_html method
  • version_check.py added
  • Error messages modified
  • Document modified
  • Website changed to http://www.pycm.ir
  • Interpretation functions moved to pycm_interpret.py
  • Utility functions moved to pycm_util.py
  • Unnecessary else and elif removed
  • == changed to is

 Source Code (.Zip) (675 KB)

 Source Code (.tar.gz) (663 KB)

 Version 1.7

  • Gini Index (GI) added
  • Example-7 added
  • pycm_profile.py added
  • class_name argument added to stat,save_stat,save_csv and save_html methods
  • overall_param and class_param arguments empty list bug fixed
  • matrix_params_calc, matrix_params_from_table and vector_filter functions optimized
  • overall_MCC_calc, CEN_misclassification_calc and convex_combination functions optimized
  • Document modified

 Source Code (.Zip) (671 KB)

 Source Code (.tar.gz) (660 KB)

 Version 1.6

  • AUC Value Interpretation (AUCI) added
  • Example-6 added
  • Anaconda cloud package added
  • overall_param and class_param arguments added to stat,save_stat and save_html methods
  • class_param argument added to save_csv method
  • _ removed from overall statistics names
  • README.md modified
  • Document modified

 Source Code (.Zip) (608 KB)

 Source Code (.tar.gz) (597 KB)

 Version 1.5

  • Relative Classifier Information (RCI) added
  • Discriminator Power (DP) added
  • Youden's Index (Y) added
  • Discriminant Power Interpretation (DPI) added
  • Positive Likelihood Ratio Interpretation (PLRI) added
  • __len__ method added
  • relabel method added
  • __class_stat_init__ function added
  • __overall_stat_init__ function added
  • matrix attribute added
  • normalized_matrix attribute added
  • normalized_table attribute added
  • Python 3.7 added to .travis.yml & appveyor.yml
  • README.md modified
  • Document modified
  • LR+ renamed to PLR
  • LR- renamed to NLR
  • normalized_matrix method renamed to print_normalized_matrix
  • matrix method renamed to print_matrix
  • entropy_calc fixed
  • cross_entropy_calc fixed
  • conditional_entropy_calc fixed
  • print_table bug for large numbers fixed
  • JSON key bug in save_obj fixed
  • transpose bug in save_obj

 Source Code (.Zip) (604 KB)

 Source Code (.tar.gz) (593 KB)

 Version 1.4

  • Area Under Curve added
  • AUNU added
  • AUNP added
  • Class Balance Accuracy added
  • Global Performance Index (RR) added
  • Overall MCC added
  • Distance index (dInd) added
  • Similarity index (sInd) added
  • one_vs_all added
  • dev-requirements.txt added
  • README.md modified
  • Document modified
  • save_stat modified
  • requirements.txt modified

 Source Code (.Zip) (597 KB)

 Source Code (.tar.gz) (586 KB)

 Version 1.3

  • Confusion Entropy added
  • Overall Confusion Entropy added
  • Modified Confusion Entropy added
  • Overall Modified Confusion Entropy added
  • Information Score added
  • README.md modified

 Source Code (.Zip) (588 KB)

 Source Code (.tar.gz) (577 KB)

 Version 1.2

  • NIR (No Information Rate) added
  • P-Value added
  • sample_weight added
  • transpose added
  • README.md modified
  • Key error in some parameters fixed
  • OSX env added to .travis.yml

 Source Code (.Zip) (584 KB)

 Source Code (.tar.gz) (574 KB)

 Version 1.1

  • Zero-one loss added
  • Support added
  • online_help function added
  • README.md modified
  • html_table function modified
  • table_print function modified
  • normalized_table_print function modified

 Source Code (.Zip) (503 KB)

 Source Code (.tar.gz) (495 KB)

 Version 1.0

  • Hamming loss added
  • README.md modified

 Source Code (.Zip) (500 KB)

 Source Code (.tar.gz) (493 KB)

 Version 0.9.5

  • Obj load added
  • Obj save added
  • Example-4 added
  • README.md modified
  • Block diagram updated

 Source Code (.Zip) (500 KB)

 Source Code (.tar.gz) (493 KB)

 Version 0.9

  • Activation Threshold added
  • Example-3 added
  • Jaccard index added
  • Overall Jaccard index added
  • README.md modified
  • setup.py modified

 Source Code (.Zip) (361 KB)

 Source Code (.tar.gz) (353 KB)

 Version 0.8.6

  • Example section in document added
  • Python 2.7 CI added
  • JOSS paper pdf added
  • Cite section modified
  • ConfusionMatrix docstring modified
  • round function changed to numpy.around
  • README.md modified

 Source Code (.Zip) (358 KB)

 Source Code (.tar.gz) (351 KB)

 Version 0.8.5

  • Confusion matrix size in HTML report modified
  • Example-1 (Comparison of three different classifiers) added
  • Example-2 (How to plot via matplotlib) added
  • ConfusionMatrix docstring added
  • JOSS paper added
  • Test system modified
  • README.md modified

 Source Code (.Zip) (237 KB)

 Source Code (.tar.gz) (227 KB)

 Version 0.8.1

  • New parameters added
    • Goodman and Kruskal's lambda B
    • Goodman and Kruskal's lambda A
    • Cross Entropy
    • Conditional Entropy
    • Joint Entropy
    • Reference Entropy
    • Response Entropy
    • Kullback-Liebler divergence
    • Mutual Information
    • Kappa Unbiased
    • Kappa No Prevalence
    • Random Accuracy Unbiased
  • Direct ConfusionMatrix added
  • pycmVectorError class added
  • pycmMatrixError class added
  • Support numpy arrays
  • pycmError class removed

 Source Code (.Zip) (111 KB)

 Source Code (.tar.gz) (105 KB)

 Version 0.7

  • New parameters added
    • Cramer's V
    • 95% Confidence interval
    • Chi-Squared
    • Phi-Squared
    • Chi-Squared DF
    • Standard error
    • Kappa standard error
    • Kappa 95% confidence interval
    • Cicchetti benchmark
  • Parameters description links in HTML report modified
  • Confusion matrix color in HTML report modified

 Source Code (.Zip) (60 KB)

 Source Code (.tar.gz) (56 KB)

 Version 0.6

  • CSV Report added (save_csv)
  • Changelog added
  • Confusion matrix color in HTML report changed
  • Parameters description links in HTML report added

 Source Code (.Zip) (56 KB)

 Source Code (.tar.gz) (52 KB)

 Version 0.5

  • New parameters added
    • Scott's pi
    • Gwet's AC1
    • Bennett S score
  • HTML Report Added (save_html)

 Source Code (.Zip) (50 KB)

 Source Code (.tar.gz) (48 KB)

 Version 0.4

  • New parameters added
    • TPR Micro/Macro
    • PPV Micro/Macro
    • RACC Overall
  • Normalized matrix bugs fixed
  • Zero class bugs fixed
  • Fleiss & Altman benchmarks added
  • Output file(.pycm) method added

 Source Code (.Zip) (47 KB)

 Source Code (.tar.gz) (45 KB)

 Version 0.3

  • New parameters added
    • Kappa
    • Random Accuracy
    • Strength of Agreement
  • Overall statistics added
  • Round bug fixed

 Source Code (.Zip) (45 KB)

 Source Code (.tar.gz) (43 KB)

 Version 0.2

  • New parameters added
    • Population
    • Condition positive
    • Test outcome positive
    • Test outcome negative
    • Prevalence
    • G-measure
  • matrix & normalized_matrix method added
  • params method added

 Source Code (.Zip) (42 KB)

 Source Code (.tar.gz) (40 KB)

 Version 0.1

  • First Release
  • Supported Class Statistics
    • ACC
    • BM
    • DOR
    • F1-Score
    • FDR
    • FNR
    • FOR
    • FPR
    • LR+
    • LR-
    • MCC
    • MK
    • NPV
    • PPV
    • TNR
    • TPR

 Source Code (.Zip) (40 KB)

 Source Code (.tar.gz) (38 KB)