Details
-
Bug
-
Status: Complete
-
High
-
Resolution: Done
-
None
-
None
-
None
-
None
-
INDY 18.01: Stability+
Description
Build Info:
indy-node 1.2.255
Steps to Reproduce:
1. Install node and stop it.
2. Place file with 100k json NYMs to node named `txns`.
3. Run `python3 -m cProfile -s time add_json_txns_to_ledger.py txns`.
4. Check the cprofile results.
Actual Results:
~5400 seconds from ~6500 are spent in kv_store.py at lines 64 and 69.
99% of all functions' calls are performed in kv_store.py at line 69.
10091189092 function calls (10086844765 primitive calls) in 6563.480 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 10003200163 3039.453 0.000 3039.453 0.000 kv_store.py:69(<genexpr>) 200016 2371.693 0.012 6481.807 0.032 kv_store.py:64(size) 200026 1064.881 0.005 4104.334 0.021 {built-in method builtins.sum} 915129 17.407 0.000 25.512 0.000 base58.py:30(b58encode) 299994 9.021 0.000 10.000 0.000 {method 'Put' of 'leveldb.LevelDB' objects} 40228222 7.379 0.000 7.379 0.000 {built-in method builtins.divmod} 400037 4.235 0.000 4.235 0.000 {method 'RangeIter' of 'leveldb.LevelDB' objects} 1600164/1600034 2.716 0.000 6.400 0.000 typing.py:1043(__subclasscheck__) 200000 2.160 0.000 8.971 0.000 msgpack_serializer.py:45(__sort_dict) 7832836/4832307 1.993 0.000 8.831 0.000 {built-in method builtins.isinstance} 1600227/1600059 1.711 0.000 3.060 0.000 abc.py:194(__subclasscheck__) 100000 1.695 0.000 6501.976 0.065 compact_merkle_tree.py:95(_push_subtree) 100000 1.669 0.000 6488.565 0.065 leveldb_hash_store.py:21(writeLeaf) 400037 1.545 0.000 5.780 0.000 kv_store_leveldb.py:33(iterator) 1330300/99998 1.463 0.000 3.362 0.000 hash_store.py:87(getNodePosition) 199994/100000 1.416 0.000 4.084 0.000 compact_merkle_tree.py:139(__push_subtree_hash) 1 1.386 1.386 6563.483 6563.483 add_json_txns_to_ledger.py:3(<module>) 100000 1.351 0.000 31.568 0.000 ledger.py:138(_build_merkle_proof) 3000559 1.347 0.000 1.347 0.000 _weakrefset.py:70(__contains__) 1630306 1.331 0.000 1.331 0.000 util.py:24(highest_bit_set) 100000 1.309 0.000 1.309 0.000 decoder.py:345(raw_decode) 200000 1.134 0.000 2.177 0.000 __init__.py:41(packb) 299994 1.103 0.000 11.728 0.000 kv_store_leveldb.py:45(put) 815122 1.075 0.000 2.752 0.000 tree_hasher.py:27(hash_children) 200000 1.043 0.000 1.043 0.000 {method 'pack' of 'msgpack._packer.Packer' objects} 1630298 0.992 0.000 0.992 0.000 util.py:5(count_bits_set) 3415 0.990 0.000 2.251 0.001 inspect.py:690(getmodule) 1788618 0.978 0.000 0.978 0.000 kv_store_leveldb_int_keys.py:13(compare) 199998 0.944 0.000 1.734 0.000 compact_merkle_tree.py:35(_update) 1600000 0.863 0.000 7.262 0.000 typing.py:1035(__instancecheck__) 200032 0.814 0.000 0.849 0.000 {built-in method builtins.sorted} 200012 0.746 0.000 6482.553 0.032 leveldb_hash_store.py:59(leafCount) 915122 0.724 0.000 0.724 0.000 {method 'update' of '_hashlib.HASH' objects} 915123 0.690 0.000 0.690 0.000 {built-in method _hashlib.openssl_sha256} 100000 0.688 0.000 6502.664 0.065 compact_merkle_tree.py:156(append) 915122 0.685 0.000 0.685 0.000 {method 'digest' of '_hashlib.HASH' objects} 915114 0.679 0.000 26.190 0.000 ledger.py:228(hashToStr) 100004 0.649 0.000 2.714 0.000 tree_hasher.py:73(_hash_fold) 200000 0.637 0.000 14.049 0.000 msgpack_serializer.py:21(serialize) 200979 0.595 0.000 0.595 0.000 {method 'match' of '_sre.SRE_Pattern' objects} 100000 0.530 0.000 6534.762 0.065 ledger.py:133(_addToTreeSerialized) 100000 0.521 0.000 22.225 0.000 ledger.py:142(<listcomp>)
Expected Results:
Investigation of actual results is needed for some optimization fixes.