CPU 코어 부하경감법 (An way to offload CPU single core load to others by Glenn)

[sword]

안녕하세요.
(Hello.)

Glenn이 validator process의 경우 CPU 코어 하나에만 부하가 집중되는 현상을 해결하는 방법을 개발했고, 시험결과 코어별 부하가 균등하게 분배되는 것을 확인하여 공유드리고자 합니다.
(Thanks to Glenn’s hard work, CPU single core load of validator process was distributed evenly over CPU cores, and I found that it works. Great work, Glenn!)

아래와 같이 하시면 됩니다.
(Here is the way.)

0. yaml파일을 백업하세요.
   (Backup your yaml file. )

cp creditcoin-with-gateway.yaml creditcoin-with-gateway.yaml.bak

1. 일단 도커 컨테이너를 종료합니다.
   (First, Terminate Docker containers. )

docker-compose -f creditcoin-with-gateway.yaml down

2. 아래 파일을 다운받아서 Server디렉토리에 넣으세요. "docker-compose.yaml"파일이나 “creditcoin-with-gateway.yaml”, “creditcoin-default.yaml” 파일과 같은 디렉토리에 위치하면 됩니다.
   (Download “init.py” and place it to Server directory. It should be same path with “docker-compose.yaml”, “creditcoin-with-gateway.yaml” or “creditcoin-default.yaml”)

https://github.com/gluwa/CreditcoinDockerCompose-Mainnet/blob/c37d64918efb9cc8e394a214920bef61fd6b937e/Server/init.py

3. 서버 기동 yaml 파일을 아래와 같이 편집합니다.
   (Edit your yaml file as following guide. )
   

• “services :” 항목의 “validator :” 항목 밑에 “secret :” 항목에 아래 두 줄을 추가합니다. “secret :” 항목이 없으면 추가하세요.
  (Add two lines as followings. If there is no “secret :” key, add it. )

- source: concurrent_pow_solvers
  target: /usr/local/lib/python3.5/dist-packages/PoW/__init__.py

• “secret :” 항목에 아래 두 줄을 추가합니다.
  (Add two lines as followings. )

concurrent_pow_solvers:
file: ./__init__.py

4. 서버를 기동합니다.
   (Start your server again. )

docker-compose -f creditcoin-with-gateway.yaml up -d

5. CPU 부하경감을 확인하세요.
   (Check your CPU load by “top” and press “1”. )

• top 실행 후 숫자 1 입력
• 

[anon7085849]

The original Proof-of-Work Python script was written by someone else:

gluwa/Creditcoin/blob/master/Creditcoin/PoW/PoW/__init__.py

I modified the Solver to run concurrently over multiple CPU cores.

[ctcboom]

에러가 뜨네요

[2020-10-18 12:58:50.101 CRITICAL publisher] on_check_publish_block exception.
[2020-10-18 12:58:50.101 CRITICAL publisher] on_check_publish_block exception.
[2020-10-18 12:58:50.101 ERROR publisher] invalid syntax (init.py, line 6)
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/publisher.py”, line 748, in on_check_publish_block
self._build_candidate_block(self._chain_head)
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/publisher.py”, line 563, in _build_candidate_block
state_view)
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/consensus/consensus_factory.py”, line 80, in get_configured_consensus_module
consensus_module_name)
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/consensus/consensus_factory.py”, line 55, in get_consensus_module
return importlib.import_module(module_package)
File “/usr/lib/python3.5/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 986, in _gcd_import
File “”, line 969, in _find_and_load
File “”, line 958, in _find_and_load_unlocked
File “”, line 673, in _load_unlocked
File “”, line 661, in exec_module
File “”, line 767, in get_code
File “”, line 727, in source_to_code
File “”, line 222, in _call_with_frames_removed
File “/usr/local/lib/python3.5/dist-packages/PoW/init.py”, line 6

^
SyntaxError: invalid syntax
[2020-10-18 12:58:50.101 ERROR publisher] invalid syntax (init.py, line 6)
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/publisher.py”, line 748, in on_check_publish_block
self._build_candidate_block(self._chain_head)
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/publisher.py”, line 563, in _build_candidate_block
state_view)
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/consensus/consensus_factory.py”, line 80, in get_configured_consensus_module
consensus_module_name)
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/consensus/consensus_factory.py”, line 55, in get_consensus_module
return importlib.import_module(module_package)
File “/usr/lib/python3.5/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 986, in _gcd_import
File “”, line 969, in _find_and_load
File “”, line 958, in _find_and_load_unlocked
File “”, line 673, in _load_unlocked
File “”, line 661, in exec_module
File “”, line 767, in get_code
File “”, line 727, in source_to_code
File “”, line 222, in _call_with_frames_removed
File “/usr/local/lib/python3.5/dist-packages/PoW/init.py”, line 6

^
SyntaxError: invalid syntax
[2020-10-18 12:58:50.205 CRITICAL publisher] on_check_publish_block exception.
[2020-10-18 12:58:50.205 CRITICAL publisher] on_check_publish_block exception.
[2020-10-18 12:58:50.205 ERROR publisher] invalid syntax (init.py, line 6)
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/publisher.py”, line 748, in on_check_publish_block
self._build_candidate_block(self._chain_head)
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/publisher.py”, line 563, in _build_candidate_block
state_view)
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/consensus/consensus_factory.py”, line 80, in get_configured_consensus_module
consensus_module_name)
File “/usr/lib/python3/dist-packages/sawtooth_validator/journal/consensus/consensus_factory.py”, line 55, in get_consensus_module
return importlib.import_module(module_package)
File “/usr/lib/python3.5/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 986, in _gcd_import
File “”, line 969, in _find_and_load
File “”, line 958, in _find_and_load_unlocked
File “”, line 673, in _load_unlocked
File “”, line 661, in exec_module
File “”, line 767, in get_code
File “”, line 727, in source_to_code
File “”, line 222, in _call_with_frames_removed
File “/usr/local/lib/python3.5/dist-packages/PoW/init.py”, line 6

[anon7085849]

Line 6 of __init__.py is a Python comment. Did you copy and paste (eg. ctrl-C and ctrl-V) from GitHub page or use ‘scp’ to transfer the .py file to your host machine? Sometimes copy and paste inadvertently capture control characters.

On Ubuntu Linux, you should:

$ git clone -b concurrent_pow_solvers https://github.com/gluwa/CreditcoinDockerCompose-Mainnet.git

and then use ‘scp’ to transfer the .py file.

[nauti16]

@sword

/usr/local/lib/python3.5/dist-packages/PoW/__init__.py

I did download “ init .py” and place it to Server directory.
But there are no python3.5 path and init.py in my docker.
I’m using Windows 10 environment.

ls /usr/local/lib/python3.5/dist-packages/PoW/init.py
ls: cannot access ‘/usr/local/lib/python3.5/dist-packages/PoW/init.py’: No such file or directory

Is that ok?

제 docker 안에 ls로 찾아보면 python3.5 폴더 자체가 없는데 그래도 문제가 없을까요?

[anon7085849]

__init__.py must be placed in the same folder as your ‘docker_compose.yaml’ file. When
> docker-compose -f docker_compose.yaml up -d
is run from a PowerShell window, the .py file is copied into the container ‘sawtooth-validator-default’ at startup.

Run this command on Ubuntu Linux to check if you have the latest Validator container:
$ docker inspect sawtooth-validator-default | grep sourceversion
“com.visualstudio.gluwa.image.build.sourceversion”: “08d788c292f47cd2e7c153e1bcb501b746154ad0”

Equivalent in PowerShell:
> docker inspect sawtooth-validator-default | findstr sourceversion

[nauti16]

I’ve checked version and it’s exactly the same as what you told.
Then I assume there is no problem.

Thanks!

[dlwlalsggg]

Can I use it on mainnet?