filecoin lotus benchmarks v26 GeForce GTX 2080 Ti GeForce GTX 1080 Ti

╰+哭是因爲堅強的太久メ 2023-03-13 13:57 32阅读 0赞

### filecoin lotus benchmarks v26 GeForce GTX 2080 Ti ###

*  1，编译bench
 *  2，基准测试
 *  3，GeForce GTX 2080 Ti 测试

# 1，编译bench #

*  版本

# git show
    commit d11e375f828526eab2a59fd6fd3db88e3c5cb9d3 (HEAD -> interopnet, tag: interop.5.13, origin/interopnet)
    Merge: 7b8ee402 1d8c8b9a
    Author: Travis Person <travisperson@users.noreply.github.com>
    Date:   Wed May 13 05:44:28 2020 +0000
    
        Merge remote-tracking branch 'origin/fix/check-gas-limits' into interopnet

*  设置环境变量

# export ALL_PROXY=socks5://127.0.0.1:1080
    # git config --global http.proxy 'socks5://127.0.0.1:1080'
    # export GO111MODULE=on
    # export GOPROXY=https://goproxy.cn

env RUSTFLAGS="-C target-cpu=native -g" FFI_BUILD_FROM_SOURCE=1 make clean deps bench

*  最后显示，编译成功，当前目录可以看到`bench`

go build -o bench ./cmd/lotus-bench
    go run github.com/GeertJohan/go.rice/rice append --exec bench -i ./build

*  `bench`版本

# ./bench --version
    2020-05-14T15:01:36.340+0800	INFO	lotus-bench	lotus-bench/main.go:74	Starting lotus-bench
    lotus-bench version 0.3.0

# 2，基准测试 #

*  设置环境变量
 *  `RUST_LOG`rust日志
 *  `FIL_PROOFS_MAXIMIZE_CACHING`最大化内存参数

# export RUST_LOG=Trace && export FIL_PROOFS_MAXIMIZE_CACHING=1

*  开始测试

# ./bench sealing --storage-dir=/storage/benchtmp --sector-size=32GiB

*  或者后台运行

# nohup ./bench sealing --storage-dir=/raid5/benchtmp --sector-size=32GiB >> /raid5/bench.log 2>&1 &

*  报错

2020-05-14T23:16:04.644 INFO filecoin_proofs::api::seal > seal_commit_phase1:end
    2020-05-14T23:16:04.711 INFO filcrypto::proofs::api > seal_commit_phase1: finish
    2020-05-14T23:16:04.805+0800	INFO	lotus-bench	lotus-bench/main.go:499	Generating PoRep for sector (2)
    2020-05-14T23:16:04.825 INFO filcrypto::proofs::api > seal_commit_phase2: start
    2020-05-14T23:16:04.900 INFO filecoin_proofs::api::seal > seal_commit_phase2:start
    2020-05-14T23:16:04.901 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
    2020-05-14T23:16:04.901 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
    2020-05-14T23:16:04.902 INFO filecoin_proofs::caches > trying parameters memory cache for: STACKED[34359738368]
    2020-05-14T23:16:04.902 INFO filecoin_proofs::caches > no params in memory cache for STACKED[34359738368]
    2020-05-14T23:16:04.904 INFO storage_proofs_core::parameter_cache > parameter set identifier for cache: layered_drgporep::PublicParams{  graph: stacked_graph::StackedGraph{ expansion_degree: 8 base_graph: drgraph::BucketGraph{ size: 1073741824; degree: 6; hasher: poseidon_hasher} }, challenges: LayerChallenges {  layers: 11, max_count: 18 }, tree: merkletree-poseidon_hasher-8-8-0 }
    2020-05-14T23:16:04.904 INFO storage_proofs_core::parameter_cache > ensuring that all ancestor directories for: "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.params" exist
    2020-05-14T23:16:04.904 INFO storage_proofs_core::parameter_cache > checking cache_path: "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.params" for parameters
    2020-05-14T23:16:07.290 INFO storage_proofs_core::parameter_cache > read parameters from cache "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.params" 
    2020-05-14T23:16:07.290 INFO filecoin_proofs::api::seal > got groth params (34359738368) while sealing
    2020-05-14T23:16:07.290 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
    2020-05-14T23:16:07.290 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
    2020-05-14T23:16:07.290 INFO filecoin_proofs::api::seal > snark_proof:start
    2020-05-14T23:16:07.298 INFO bellperson::groth16::prover > Bellperson 0.7.0 is being used!
    2020-05-14T23:20:18.490 INFO bellperson::gpu::locks > GPU is available for FFT!
    2020-05-14T23:20:18.490 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
    2020-05-14T23:20:18.496 DEBUG bellperson::gpu::locks > GPU lock acquired!
    2020-05-14T23:20:18.732 INFO bellperson::gpu::fft > FFT: 1 working device(s) selected.
    2020-05-14T23:20:18.732 INFO bellperson::gpu::fft > FFT: Device 0: GeForce RTX 2080 Ti
    2020-05-14T23:20:18.732 INFO bellperson::domain > GPU FFT kernel instantiated!
    2020-05-14T23:23:12.352 DEBUG bellperson::gpu::locks > GPU lock released!
    2020-05-14T23:24:13.677 INFO bellperson::gpu::locks > GPU is available for Multiexp!
    2020-05-14T23:24:13.677 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
    2020-05-14T23:24:13.840 DEBUG bellperson::gpu::locks > GPU lock acquired!
    2020-05-14T23:24:13.933 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0)
    2020-05-14T23:24:13.933 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 2080 Ti (Chunk-size: 7488999)
    2020-05-14T23:24:13.933 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!
    2020-05-14T23:50:07.906 DEBUG bellperson::gpu::locks > GPU lock released!
    2020-05-14T23:50:09.849 INFO filecoin_proofs::api::seal > snark_proof:finish
    2020-05-14T23:50:09.849 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
    2020-05-14T23:50:09.849 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
    2020-05-14T23:50:09.849 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
    2020-05-14T23:50:09.849 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
    2020-05-14T23:50:09.849 INFO filecoin_proofs::caches > trying parameters memory cache for: STACKED[34359738368]-verifying-key
    2020-05-14T23:50:09.849 INFO filecoin_proofs::caches > no params in memory cache for STACKED[34359738368]-verifying-key
    2020-05-14T23:50:09.850 INFO storage_proofs_core::parameter_cache > parameter set identifier for cache: layered_drgporep::PublicParams{  graph: stacked_graph::StackedGraph{ expansion_degree: 8 base_graph: drgraph::BucketGraph{ size: 1073741824; degree: 6; hasher: poseidon_hasher} }, challenges: LayerChallenges {  layers: 11, max_count: 18 }, tree: merkletree-poseidon_hasher-8-8-0 }
    2020-05-14T23:50:09.850 INFO storage_proofs_core::parameter_cache > ensuring that all ancestor directories for: "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.vk" exist
    2020-05-14T23:50:09.850 INFO storage_proofs_core::parameter_cache > checking cache_path: "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.vk" for verifying key
    2020-05-14T23:50:09.946 INFO storage_proofs_core::parameter_cache > read verifying key from cache "/storage/filecoin-proof-parameters-v26/parameters-v26/v26-stacked-proof-of-replication-merkletree-poseidon_hasher-8-8-0-sha256_hasher-82a357d2f2ca81dc61bb45f4a762807aedee1b0a53fd6c4e77b46a01bfef7820.vk" 
    2020-05-14T23:50:09.946 INFO filecoin_proofs::api::seal > got verifying key (34359738368) while verifying seal
    2020-05-14T23:50:09.971 INFO bellperson::gpu::locks > GPU is available for Multiexp!
    2020-05-14T23:50:09.971 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
    2020-05-14T23:50:10.023 DEBUG bellperson::gpu::locks > GPU lock acquired!
    2020-05-14T23:50:10.123 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0)
    2020-05-14T23:50:10.124 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 2080 Ti (Chunk-size: 7488999)
    2020-05-14T23:50:10.124 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!
    2020-05-14T23:50:10.219 DEBUG bellperson::gpu::locks > GPU lock released!
    2020-05-14T23:50:10.219 INFO filecoin_proofs::api::seal > seal_commit_phase2:end
    2020-05-14T23:50:10.219 INFO filcrypto::proofs::api > seal_commit_phase2: finish
    2020-05-14T23:50:10.219 INFO filcrypto::proofs::api > verify_seal: start
    2020-05-14T23:50:10.220 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
    2020-05-14T23:50:10.220 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
    2020-05-14T23:50:10.220 INFO storage_proofs_porep::stacked::vanilla::graph > using parents cache of unlimited size
    2020-05-14T23:50:10.220 INFO storage_proofs_porep::stacked::vanilla::graph > using parent_cache[1073741824]
    2020-05-14T23:50:10.220 INFO filecoin_proofs::caches > trying parameters memory cache for: STACKED[34359738368]-verifying-key
    2020-05-14T23:50:10.220 INFO filecoin_proofs::caches > found params in memory cache for STACKED[34359738368]-verifying-key
    2020-05-14T23:50:10.220 INFO filecoin_proofs::api::seal > got verifying key (34359738368) while verifying seal
    2020-05-14T23:50:10.237 INFO bellperson::gpu::locks > GPU is available for Multiexp!
    2020-05-14T23:50:10.237 DEBUG bellperson::gpu::locks > Acquiring GPU lock...
    2020-05-14T23:50:10.414 DEBUG bellperson::gpu::locks > GPU lock acquired!
    2020-05-14T23:50:10.522 INFO bellperson::gpu::multiexp > Multiexp: 1 working device(s) selected. (CPU utilization: 0)
    2020-05-14T23:50:10.522 INFO bellperson::gpu::multiexp > Multiexp: Device 0: GeForce RTX 2080 Ti (Chunk-size: 7488999)
    2020-05-14T23:50:10.522 INFO bellperson::multiexp > GPU Multiexp kernel instantiated!
    2020-05-14T23:50:10.612 DEBUG bellperson::gpu::locks > GPU lock released!
    2020-05-14T23:50:10.612 INFO filcrypto::proofs::api > verify_seal: finish
    2020-05-14T23:50:22.791+0800	WARN	lotus-bench	lotus-bench/main.go:90	failed to run seals:
        main.glob..func3
            /root/interopnet/lotus/cmd/lotus-bench/main.go:233
      - porep proof for sector 1 was invalid:
        main.runSeals
            /root/interopnet/lotus/cmd/lotus-bench/main.go:550

# 3，GeForce GTX 2080 Ti 测试 #

*  CPU `3950X`

# lscpu
    AMD Ryzen 9 3950X 16-Core Processor

*  显卡 `GeForce GTX 1080 Ti`

# lspci | grep -i nvidia
    2d:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1)
    2d:00.1 Audio device: NVIDIA Corporation GP102 HDMI Audio Controller (rev a1)

*  内存，交换分区

# free -h
                  总计         已用        空闲      共享    缓冲/缓存    可用
    内存：       125Gi       729Mi       104Gi       0.0Ki        20Gi       123Gi
    交换：       105Gi       302Mi       105Gi

*  bench 编译版本

# git show
    commit 54b3eafd6d541a68199965bbf13ec2e339c625f5 (HEAD -> master, origin/master, origin/HEAD)
    Merge: 5c73602d f6589407
    Author: Whyrusleeping <why@ipfs.io>
    Date:   Fri May 15 13:05:27 2020 -0700
    
        Merge pull request #1758 from filecoin-project/fix/lotus-bench-secp
    
        register secp signatures for lotus bench usage

*  bench编译

env RUSTFLAGS="-C target-cpu=native -g" FFI_BUILD_FROM_SOURCE=1 make clean deps bench

*  bench 运行

# export RUST_LOG=Trace && export FIL_PROOFS_MAXIMIZE_CACHING=1

nohup ./bench sealing --storage-dir=/raid5/benchtmp --sector-size=32GiB >> /raid5/bench.log 2>&1 &

*  bench 测试结果，跑完大概12个小时

results (v26) (34359738368)
    seal: addPiece: 6m26.577337189s (84.8 MiB/s)
    seal: preCommit phase 1: 4h17m35.603586994s (2.12 MiB/s)
    seal: preCommit phase 2: 1h43m21.981600421s (5.28 MiB/s)
    seal: commit phase 1: 1m4.253378137s (510 MiB/s)
    seal: commit phase 2: 1h2m37.451786187s (8.72 MiB/s)
    seal: verify: 37.58607ms
    unseal: 4h17m51.34613723s  (2.12 MiB/s)
    
    generate candidates: 7.194132ms (4.34 TiB/s)
    compute winnnig post proof (cold): 14.527132336s
    compute winnnig post proof (hot): 6.148464516s
    verify winnnig post proof (cold): 98.07689ms
    verify winnnig post proof (hot): 14.109101ms
    
    compute window post proof (cold): 15m57.884786147s
    compute window post proof (hot): 16m16.748839396s
    verify window post proof (cold): 5.504320785s
    verify window post proof (hot): 44.248935ms

*  ben运行日志
 *  [bench完整运行日志][bench]

参考：

1.  [filecoin-project benchmarks][]
2.  [filecoin benchmarks v25][]
3.  [CoinSummer实验室Filecoin资源分享][CoinSummer_Filecoin]
4.  [轻松共享大量代码][Link 1]

[bench]: https://www.codepile.net/pile/dPKlAKW0
[filecoin-project benchmarks]: https://github.com/filecoin-project/benchmarks
[filecoin benchmarks v25]: https://blog.csdn.net/u010953692/article/details/105674010
[CoinSummer_Filecoin]: https://github.com/CoinSummer/filecoin
[Link 1]: https://www.codepile.ne