amd Archives - Efison Lisan Teknologi https://efisonlt.com/tag/amd/ Computation for Everybody Fri, 14 Aug 2026 03:54:44 +0000 en-US hourly 1 https://wordpress.org/?v=7.0.4 https://efisonlt.com/wp-content/uploads/2020/03/cropped-efison_logo_orange-skuer-32x32.png amd Archives - Efison Lisan Teknologi https://efisonlt.com/tag/amd/ 32 32 Our Experience with Asus AMD Radeon AI Pro R9700 Turbo https://efisonlt.com/our-experience-with-asus-amd-radeon-ai-pro-r9700-turbo/?utm_source=rss&utm_medium=rss&utm_campaign=our-experience-with-asus-amd-radeon-ai-pro-r9700-turbo Sat, 11 Oct 2025 13:13:19 +0000 https://efisonlt.com/?p=1880 2025, and AI. What’s not to love? Again, if somebody were to sell a rendang and they state that it was created using AI, I bet venture capitals would clap and circle like vultures. Okay enough yapping. Now we are talking about a damn GPU. A tool to run real AI. Introducing, AMD Radeon AI… Read More »Our Experience with Asus AMD Radeon AI Pro R9700 Turbo

The post Our Experience with Asus AMD Radeon AI Pro R9700 Turbo appeared first on Efison Lisan Teknologi.

]]> 2025, and AI.

What’s not to love?

Again, if somebody were to sell a rendang and they state that it was created using AI, I bet venture capitals would clap and circle like vultures.

Okay enough yapping. Now we are talking about a damn GPU. A tool to run real AI.

Introducing, AMD Radeon AI Pro R9700

In their wisdom, calling a prosumer/workstation GPU with a “pro” moniker doesn’t quite cut it anymore. It now has AI in its name. Why, do you ask?

Because apparently it has 32GB worth of VRAM and it supports another numerical precision (FP8). The VRAM is quite big, we reckon. Still not as big as the last gen Pro (no AI) W7900, but with all its new-ness, AI-ness, and goodness, it’s still pretty nice.

Radeon AI Pro R9700Radeon Pro W7900
ArchitectureAMD RDNA 4AMD RDNA 3
Memory size32 GB48 GB
Memory bandwidth644.6 GB/s864 GB/s
Memory ECC supportYes (Linux only)*Yes
Peak FP32 (vector) performance47.8 TFLOPS61.3 TFLOPS
Peak FP16 (vector) performance95.7 TFLOPS123 TFLOPS
Peak FP16 (matrix) performance191 TFLOPS123 TFLOPS
Peak FP8 (matrix) performance383 TFLOPSN/A
Peak INT8 (matrix) performance383 TOPS123 TOPS

Personally we’re pretty confused about why the memory ECC support on R9700 is stated as Linux only but as we’re mostly using Linux as our test platform, no complaint there.

Even with smaller memory size, it is pretty beefy in terms of performance. Especially in theoretical matrix performance. Hence the AI namesake. Unfortunately we have no W7900 or any RX 7900 XTX for real-world comparison purpose, but we do have an RTX 5070 Ti which apparently has a similar profile.

Radeon AI Pro R9700Geforce RTX 5070 Ti
Process technologyTSMC N4PTSMC 4N
Die size357 mm²378 mm²
Memory size32 GB GDDR616 GB GDDR7
Memory interface256-bit256-bit
Memory bandwidth644.6 GB/s896 GB/s
Total board power300 W300 W

The Test Setup

ComponentSpecification
CPUIntel Core i7-12700K
@5.0 GHz
MotherboardASRock Z690 PG Velocita
Memory4*24 GB Klevv Cras V RGB
@DDR5-5600
Storage2TB MSI Spatium M480
PSU1000W 1stPlayer NGDP Gold

For the benchmark, we tested LLM using llama.cpp and image generation using Qwen Image Edit 2509.

llama.cpp Benchmark Setup

We used llama.cpp build d2ee056e1 (6713) and compiled the CPU backend using Intel oneAPI compiler 2025.2.1 against external BLAS library which is Intel oneAPI MKL 2025.2. Why, do you ask? Because it yields faster performance compared to mere GNU compiler 15.2.1 with no BLAS.

We tested using unsloth/gpt-oss-120b-Q4_K_M model and .(7|8|9|[0-9][0-9]|[0-9][0-9][0-9]).ffn_(up|down|gate)_exps. MoE layers which are then being put to system RAM for CPU offload processing.

(In token/s. Higher is better)GNU compiler 15.2.1
no BLAS		oneAPI compiler 2025.2.1
BLAS=oneAPI MKL 2025.2
Prompt processing (512 tokens)180.65 ± 1.74182.39 ± 1.60
Text generation (256 tokens)21.85 ± 0.8232.19 ± 0.04
Compilation steps for GNU compiler 15.2.1, no BLAS
Bash
mkdir build_no-blas-gcc_vulkan && cd build_no-blas-gcc_vulkan
cmake .. -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DGGML_NATIVE=ON -DGGML_VULKAN=1
cmake --build . --config Release -j
Compilation steps for oneAPI compiler 2025.2.1, BLAS=oneAPI MKL 2025.2
Bash
mkdir build_mkl-ilp64-icx_vulkan && build_mkl-ilp64-icx_vulkan
cmake -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=Intel10_64ilp -DGGML_NATIVE=ON -DGGML_VULKAN=1
cmake --build . --config Release -j
Run output
Bash
GGML_VULKAN_DEVICE=0 ./build_no-blas-gcc_vulkan/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Graphics (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | ngl | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan     |  99 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           pp512 |        180.65 ± 1.74 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan     |  99 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           tg256 |         21.85 ± 0.82 |

build: d2ee056e1 (6713)
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Graphics (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           pp512 |        182.39 ± 1.60 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           tg256 |         32.19 ± 0.04 |

build: d2ee056e1 (6713)

For the GPU backend, we used both Vulkan and HIP (ROCm 7) which we won’t discuss much here to prevent spoilers.

We refrained on using ROCm 6.4.x as ROCm 7.0.x is now performing much better on this GPU (or probably all AMD RDNA4 GPUs in general). If you haven’t heard already, AMD have just released their newest ROCm 7 on September 16, 2025. We did a quick comparison in terms of llama.cpp performance against ROCm 6.4.3 which can be seen on this guy’s Facebook post.

As for the RTX 5070 Ti, we compiled llama.cpp GPU backend against CUDA without GGML_CUDA_FORCE_CUBLAS.

So for references, here are the compilation lines used for various configurations stated above:

Compilation line for llama.cpp ROCm
Bash
cmake .. -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=Intel10_64ilp -DGGML_NATIVE=ON -DGGML_HIP=ON -DGPU_TARGETS=gfx1201
Compilation line for llama.cpp Vulkan
Bash
cmake .. -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=Intel10_64ilp -DGGML_NATIVE=ON -DGGML_VULKAN=1
Compilation line for llama.cpp CUDA
Bash
cmake .. -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=Intel10_64ilp -DGGML_NATIVE=ON -DGGML_CUDA=ON

We chose 2 models to be used for testing:

Qwen Image Edit 2509 Benchmark Setup (and rambling about PyTorch for ROCm on Windows situation)

This one is a bit different because it has PyTorch dependencies and it’s not that simple. Historically, AMD has been neglecting (or unable to make?) Windows PyTorch package for their GPUs. With the Windows Preview Edition 25.20.01.14 driver, they finally support PyTorch on Windows for Radeons. Yay!

Albeit the limited roster of supported GPUs…

Funnily enough, they only listed Windows 11 as the compatible OS. Lo and behold, we managed to use it on Windows 10.

Enough with the rambling, now let’s get onto the setup.

We did the tests on 2 different OSes:

  • Aurora Linux 42 based on Fedora Kinoite
  • Windows 10 Pro

For the PyTorch, we tested different combinations:

PyTorch 2.x for ROCm 6.4.xPyTorch 2.x for ROCm 7.0.x
Aurora Linux 42✘✔
Windows 10 Pro✔✔

The Qwen Image Edit 2509 models used here were QuantStack/Qwen-Image-Edit-2509-Q4_K_M. We also used lightx2v/Qwen-Image-Edit-2509-Lightning-4steps-V1.0-bf16 LoRA for performance reason. As for the UI and additional package, we used ComfyUI v0.3.64. Not to forgot to mention city96/ComfyUI-GGUF for GGUF models compatibility.

Additional notes about Qwen Image Edit 2509 test setup and deployment

As for the inputs, we used these images:

And these sentences for the prompt:

Test Results: llama.cpp

We talked a bit about llama.cpp LLM MoE layer offload to CPU above. But in short, CPU offloading is done to make sure you can run a big model without the big performance hit caused by memory spill from VRAM to system RAM.

So, if you run a big model on GPU, and the VRAM is smaller than the model, it will still run. But it will run terribly because now the GPU needs to access the data from RAM in which the available VRAM can’t contain, while the CPU is doing nothing to help the processing.

unsloth/gpt-oss-120b-Q4_K_M has the size of 58.45 GB, while the available VRAM is only around 32 GB before substracting other necessary services for the operating system, which is now spilling around 26-27 GB to system RAM.

Now see the difference with CPU offloading.

With .(4|5|6|7|8|9|[0-9][0-9]|[0-9][0-9][0-9]).ffn_(up|down)_exps. layers being offloaded to CPU and put to the system RAM, now the CPU can help with the processing while preventing the layers loaded to VRAM to spill over.

Probably this article is not the best at explaining how the MoE layers work or the way they’re offloaded. You can read a little bit more technical stuffs starting from reading this guide, or maybe a little bit of Google-fu. Sorry.

Now, let’s see the performance difference:

(In token/s. Higher is better)Without CPU offloadingWith CPU offloading
Prompt processing (512 tokens)120.48 ± 4.06215.93 ± 1.42
Text generation (256 tokens)11.21 ± 0.0234.97 ± 0.15
Run output
Bash
GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(4|5|6|7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Graphics (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           pp512 |        120.48 ± 4.06 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           tg256 |         11.21 ± 0.02 |

build: d2ee056e1 (6713)
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Graphics (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(4|5|6|7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           pp512 |        215.93 ± 1.42 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(4|5|6|7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           tg256 |         34.97 ± 0.15 |

build: d2ee056e1 (6713)

Let’s continue with the test results.

Here we have different configs of layers for CPU offloading. One is to keep the VRAM usage to under 16 GB, and then maximize the possible 32 GB. The configuration details can be seen below:

  • RTX 5070 Ti with just under 16 GB VRAM load, CUDA backend
  • RTX 5070 Ti with just under 16 GB VRAM load, Vulkan backend
  • R9700 with just under 16 GB VRAM load, ROCm 7 backend
  • R9700 with just under 16 GB VRAM load, Vulkan backend
  • R9700 with just under 32 GB VRAM load, ROCm 7 backend
  • R9700 with just under 32 GB VRAM load, Vulkan backend

Here are the results using unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS:

(In token/s. Higher is better)5070 Ti, 16G
CUDA		5070 Ti, 16G
Vulkan		R9700, 16G
ROCm 7		R9700, 16G
Vulkan		R9700, 32G
ROCm 7		R9700, 32G
Vulkan
Prompt processing (512 tokens)3723.33 ± 50.092739.83 ± 30.37746.77 ± 3.451236.97 ± 9.19797.92 ± 3.611665.47 ± 5.95
Text generation (256 tokens)137.34 ± 0.69138.41 ± 0.7988.98 ± 0.16105.35 ± 0.75100.05 ± 0.08122.63 ± 0.52

Here are the results using unsloth/gpt-oss-120b-Q4_K_M:

(In token/s. Higher is better)5070 Ti, 16G
CUDA		5070 Ti, 16G
Vulkan		R9700, 16G
ROCm 7		R9700, 16G
Vulkan		R9700, 32G
ROCm 7		R9700, 32G
Vulkan
Prompt processing (512 tokens)370.32 ± 4.22206.81 ± 3.52188.32 ± 4.82169.56 ± 2.60251.93 ± 6.61230.01 ± 2.78
Text generation (256 tokens)40.24 ± 0.1337.90 ± 0.4232.59 ± 0.0131.49 ± 0.0438.73 ± 0.0836.22 ± 0.03
RTX 5070 Ti 16G run output
Bash
./build_mkl-ilp64-icx_cuda/bin/llama-bench --model ../MoE/unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096

./build_mkl-ilp64-icx_cuda/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 5070 Ti, compute capability 12.0, VMM: yes
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | CUDA,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU |           pp512 |      3723.33 ± 50.09 |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | CUDA,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU |           tg256 |        137.34 ± 0.69 |

build: d2ee056e1 (6713)
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = NVIDIA GeForce RTX 5070 Ti (NVIDIA) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: NV_coopmat2
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU |           pp512 |      2739.83 ± 30.37 |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU |           tg256 |        138.41 ± 0.79 |

build: d2ee056e1 (6713)
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 5070 Ti, compute capability 12.0, VMM: yes
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | CUDA,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           pp512 |        370.32 ± 4.22 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | CUDA,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           tg256 |         40.24 ± 0.13 |

build: d2ee056e1 (6713)
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = NVIDIA GeForce RTX 5070 Ti (NVIDIA) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: NV_coopmat2
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           pp512 |        206.81 ± 3.52 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           tg256 |         37.90 ± 0.42 |

build: d2ee056e1 (6713)
R9700 16G run output
Bash
./build_mkl-ilp64-icx_rocm/bin/llama-bench --model ../MoE/unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096

./build_mkl-ilp64-icx_rocm/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU |           pp512 |        746.77 ± 3.45 |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU |           tg256 |         88.98 ± 0.16 |

build: d2ee056e1 (6713)
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU |           pp512 |       1236.97 ± 9.19 |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(39|[4-9][0-9]|[1-9][0-9][0-9])\.ffn_(gate)_exps.=CPU |           tg256 |        105.35 ± 0.75 |

build: d2ee056e1 (6713)
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           pp512 |        188.32 ± 4.82 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           tg256 |         32.59 ± 0.01 |

build: d2ee056e1 (6713)
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           pp512 |        169.56 ± 2.60 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down|gate)_exps.=CPU |           tg256 |         31.49 ± 0.04 |

build: d2ee056e1 (6713)
R9700 32G run output
Bash
./build_mkl-ilp64-icx_rocm/bin/llama-bench --model ../MoE/unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -p 512 -n 256 -fa 1 -ub 4096 -b 4096

./build_mkl-ilp64-icx_rocm/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0 --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0 --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           pp512 |        797.92 ± 3.61 |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           tg256 |        100.05 ± 0.08 |

build: d2ee056e1 (6713)
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           pp512 |       1665.47 ± 5.95 |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           tg256 |        122.63 ± 0.52 |

build: d2ee056e1 (6713)
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           pp512 |        251.93 ± 6.61 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           tg256 |         38.73 ± 0.08 |

build: d2ee056e1 (6713)
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           pp512 |        230.01 ± 2.78 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           tg256 |         36.22 ± 0.03 |

build: d2ee056e1 (6713)

Even though it has twice the VRAM, with the MoE layers CPU offloading strategy, turns out it still can’t quite compete with the RTX 5070 Ti. This might be caused by the large memory bandwidth gap between both (644.6 GB/s vs 896 GB/s, 39% difference).

Now, when you see on unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS, it has this big performance delta between ROCm 7 backend and Vulkan backend. I wonder why a bunch of open-source developers from Vulkan/SPIR-V/Kompute project can outperform engineers who are paid to make their stuffs performant… Although, it did better on unsloth/gpt-oss-120b-Q4_K_M by a tiny margin.

Also, turns out having a bigger VRAM still can’t quite defeat the smaller VRAM, if your bandwidth is too slow, and your software isn’t up to par.

It’s not about the size. It’s about how you use it.

Test Results: Qwen Image Edit 2509

Sometimes we question ourselves why do we like to suffer.

And this test is no different.

We (or me, personally), wonder if being a goose farmer is a better choice for living a happy life.

First, let us tell you that running this test for the first run, is not a happy feat. It takes quite a while for the text encoder and VAE to load and process our prompt and image inputs.

On a bad day, it can take more than 7 mins of time to generate this for the first run.

UPDATE: IT DOESN’T HAVE TO BE 7 MINUTES

PyTorch for ROCm apparently has a bug in which the VAE stage is extremely slow.

The solution? ComfyUI recently pushed a workaround. Which is to disable cuDNN back-end (?).

Now the first generation isn’t that painful anymore.

Original article continues below.

Here, we only put the generation time after text encoder and VAE had been loaded and then run it in a batch of 5. Then we averaged the time needed to generate the same prompt with the same inputs.

Now, let’s get onto the results:

(In seconds, Lower is better)RTX 5070 Ti
CUDA 12.9
Linux		R9700
ROCm 7.0.x
Linux		R9700
ROCm 6.4.x
Windows		R9700
ROCm 7.0.x
Windows
Results29.38452.1769.26262.59
RTX 5070 Ti CUDA 12.9 Linux test screenshot
R9700 ROCm 7.0.x Linux test screenshot
R9700 ROCm 6.4.x Windows test screenshot
R9700 ROCm 7.0.x Windows test screenshot

Oops.

Almost twice faster.

Also you can make a case that PyTorch for ROCm on Windows is slower. Almost 20% slower than on Linux. Although this is only on one use case that we tested and we didn’t confirm with different various use cases.

And to make it worse, RTX 5070 Ti does have a trick up its sleeve.

Introducing Nunchaku SVDQuant. An inference engine so fast it cut the inference time of Qwen Image Edit 2509 almost in half even versus already quantized Q4_K_M model.

(In seconds, Lower is better)RTX 5070 Ti
CUDA 12.9
Linux
Nunchaku FP4 r32		RTX 5070 Ti
CUDA 12.9
Linux
Q4_K_M GGUF
Results15.19829.384
RTX 5070 Ti CUDA 12.9 Linux, Nunchaku FP4 r32 model test screenshot

Overclocking

Real men do OC.

Or men with too much times in hands.

We managed to overclock this card using LACT. No, we didn’t test overclocking on Windows. Penguins FTW!

We found out that 2800 MHz was the ideal maximum VRAM clock to keep it from crashing doing various workloads. Still a healthy 282 MHz increase from the default 2518 MHz. Which translates to 11.2% more memory bandwidth.

The maximum power usage limit also increased to 330 W from 300 W (10% increase). And maybe you wonder what does GPU voltage offset about? Why is it being lowered, right?

Some of you might have heard that overclocking the GPU clock on RDNA4 is done by shifting the voltage/frequency curve by setting the voltage offset to a lower value, so the GPU would be tricked and boosts to higher frequency.

Too difficult to understand? Let us show you a video from our friend Alva Jonathan who did an excellent job explaining overclocking on another RDNA4 GPU which is a Radeon RX 9070.

Here are the performance results to show you the gain from the overclocking attempt:

llama.cpp unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS (Overclocked)

(In token/s. Higher is better)5070 Ti, 16G
CUDA		R9700, 32G
Vulkan
Overclocked		R9700, 32G
Vulkan
Stock default
Prompt processing (512 tokens)3723.33 ± 50.091810.00 ± 11.361665.47 ± 5.95
Text generation (256 tokens)137.34 ± 0.69131.09 ± 0.37122.63 ± 0.52

llama.cpp unsloth/gpt-oss-120b-Q4_K_M (Overclocked)

(In token/s. Higher is better)5070 Ti, 16G
CUDA		R9700, 32G
ROCm 7
Overclocked		R9700, 32G
ROCm 7
Stock default
Prompt processing (512 tokens)370.32 ± 4.22254.46 ± 4.62251.93 ± 6.61
Text generation (256 tokens)40.24 ± 0.1339.72 ± 0.0438.73 ± 0.08
Overclocked R9700 llama.cpp run output
Bash
./build_mkl-ilp64-icx_rocm/bin/llama-bench --model ../MoE/unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/Qwen3-Coder-30B-A3B-Instruct-1M-IQ4_XS.gguf -ctk q8_0 -ctv q8_0  --threads 8 -ngl 99 -p 512 -n 256 -fa 1 -ub 4096 -b 4096

./build_mkl-ilp64-icx_rocm/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0 --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096

GGML_VULKAN_DEVICE=0 ./build_mkl-ilp64-icx_vulkan/bin/llama-bench --model ../MoE/unsloth/gpt-oss-120b-Q4_K_M.gguf -ctk q8_0 -ctv q8_0 --threads 8 -ngl 99 -ot "\.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU" -p 512 -n 256 -fa 1 -ub 4096 -b 4096
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           pp512 |        834.27 ± 3.99 |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           tg256 |        105.42 ± 0.12 |

build: d2ee056e1 (6713)
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           pp512 |      1810.00 ± 11.36 |
| qwen3moe 30B.A3B IQ4_XS - 4.25 bpw |  15.25 GiB |    30.53 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 |           tg256 |        131.09 ± 0.37 |

build: d2ee056e1 (6713)
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           pp512 |        254.46 ± 4.62 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | ROCm,BLAS  |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           tg256 |         39.72 ± 0.04 |

build: d2ee056e1 (6713)
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = Intel(R) UHD Graphics 770 (ADL-S GT1) (Intel open-source Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
| model                          |       size |     params | backend    | threads | n_batch | n_ubatch | type_k | type_v | fa | ot                    |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | -------: | -----: | -----: | -: | --------------------- | --------------: | -------------------: |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           pp512 |        236.58 ± 2.89 |
| gpt-oss 120B Q4_K - Medium     |  58.45 GiB |   116.83 B | Vulkan,BLAS |       8 |    4096 |     4096 |   q8_0 |   q8_0 |  1 | \.(7|8|9|[0-9][0-9]|[0-9][0-9][0-9])\.ffn_(up|down)_exps.=CPU |           tg256 |         37.41 ± 0.04 |

build: d2ee056e1 (6713)

As we can see, the increase is bigger when the model is smaller, as bigger portion of layers fit on the VRAM hence the performance gain from VRAM overclocking. Still a tad slower than RTX 5070 Ti, but we’ll grab what we can.

Qwen Image Edit 2509 Q4_K_M GGUF (Overclocked)

(In seconds, Lower is better)RTX 5070 Ti
CUDA 12.9
Linux		R9700
ROCm 7.0.x
Linux
Overclocked		R9700
ROCm 7.0.x
Linux
Stock default
Results29.38448.62852.17
Overclocked R9700 run screenshot

https://efisonlt.com/wp-content/uploads/2025/10/R9700-GGUF-OC.png

7.28% faster. Not bad for a free performance gain.

Verdict

So, what do we think?

Bigger VRAM doesn’t always translate to a bigger performance. Especially in our tests which are very inference heavy. This doesn’t mean this card is DoA or something (pls don’t be, we need alternatives to the leather jacketed overlord).

There are many good takeaways. Such as AMD GPU software team are now finally getting their stuffs together. PyTorch for ROCm on Windows was one of the biggest to-do list. ROCm 7 is being faster than ROCm 6 (at least on RDNA4).

They also managed to lower the size of ROCm Docker image and PyTorch package.

Unfortunately, turns out it was due to they chose to drop RDNA2 from their supported GPUs. Sad.

AMD Radeon ROCm 7 supported GPUs

We still think there would be a better value for a bigger VRAM for different use cases. Probably in LLM finetuning, in 3D modelling/rendering, which we presume wouldn’t be easier to work around with CPU offload the way we tested MoE LLM inference above.

The post Our Experience with Asus AMD Radeon AI Pro R9700 Turbo appeared first on Efison Lisan Teknologi.

]]> EFISON EUREKA! Hadir Kembali di 2022 untuk Peneliti Indonesia! https://efisonlt.com/efison-eureka-periode-2/?utm_source=rss&utm_medium=rss&utm_campaign=efison-eureka-periode-2 Fri, 04 Mar 2022 09:28:42 +0000 https://efisonlt.com/?p=1665 Salam sehat dan sejahtera. Semoga kita semua selalu diberi keselamatan, kesehatan, dan kebahagiaan. Melanjutkan program EFISON EUREKA! yang sebelumnya pernah diluncurkan di Agustus 2021 lalu, kali ini EFISON dan AMD kembali bersama-sama menghadirkan EFISON EUREKA! periode 2. Program sponsor penggunaan HPC ini akan melanjutkan mimpi besar kita untuk memajukan ilmu pengetahuan dan riset di Indonesia.… Read More »EFISON EUREKA! Hadir Kembali di 2022 untuk Peneliti Indonesia!

The post EFISON EUREKA! Hadir Kembali di 2022 untuk Peneliti Indonesia! appeared first on Efison Lisan Teknologi.

]]> Salam sehat dan sejahtera. Semoga kita semua selalu diberi keselamatan, kesehatan, dan kebahagiaan.

Melanjutkan program EFISON EUREKA! yang sebelumnya pernah diluncurkan di Agustus 2021 lalu, kali ini EFISON dan AMD kembali bersama-sama menghadirkan EFISON EUREKA! periode 2. Program sponsor penggunaan HPC ini akan melanjutkan mimpi besar kita untuk memajukan ilmu pengetahuan dan riset di Indonesia.

Pemenang sponsor nantinya berkesempatan menggunakan ALELEON Supercomputer bertenaga AMD EPYC yang kencang dan andal. Harapannya, dengan menggunakan ALELEON Supercomputer peneliti bisa menghasilkan keluaran hasil penelitian yang berkualitas dan bertaraf internasional. Perlu diketahui pula bahwa peserta dan pemenang sponsor EFISON EUREKA! tidak dipungut biaya.

Rekan-rekan peneliti yang sudah mempersiapkan penelitian yang akan ataupun sedang dilakukan bisa langsung mencoba mendaftar melalui formulir Pendaftaran Program Sponsor HPC EFISON EUREKA!. Pendaftaran dibuka mulai dari 4 Maret 2022 hingga 25 Maret 2022.

Informasi lebih lanjut bisa dibaca di wiki EUREKA! Periode 2. Berbagai pertanyaan mengenai program sponsor ini bisa langsung ditanyakan di kontak resmi sponsorship@efisonlt.com.

Mari kita bersama-sama maju dengan teknologi superkomputer dan memperluas jangkauan komputasi HPC di Indonesia.

Salam,

Wilson Lisan
CEO Efison Lisan Teknologi

The post EFISON EUREKA! Hadir Kembali di 2022 untuk Peneliti Indonesia! appeared first on Efison Lisan Teknologi.

]]> Review ASUS ESC4000A-E10, server 2U dengan AMD EPYC dan Nvidia Tesla untuk HPC https://efisonlt.com/review-asus-esc4000a-e10/?utm_source=rss&utm_medium=rss&utm_campaign=review-asus-esc4000a-e10 Mon, 23 Nov 2020 10:16:48 +0000 https://efisonlt.com/?p=889 ASUS telah dikenal oleh umum sebagai produsen berbagai produk konsumer berkualitas seperti gadget, laptop, maupun komponen PC. Mengikuti perkembangan permintaan pasar akan komputasi kelas enterprise, saat ini ASUS mulai menapaki pasar server dengan lebih serius. Kali ini EFISON kedatangan server ASUS ESC4000A-E10 yang ditenagai CPU AMD EPYC Rome dengan arsitektur AMD Zen 2 dan GPU… Read More »Review ASUS ESC4000A-E10, server 2U dengan AMD EPYC dan Nvidia Tesla untuk HPC

The post Review ASUS ESC4000A-E10, server 2U dengan AMD EPYC dan Nvidia Tesla untuk HPC appeared first on Efison Lisan Teknologi.

]]> ASUS telah dikenal oleh umum sebagai produsen berbagai produk konsumer berkualitas seperti gadget, laptop, maupun komponen PC. Mengikuti perkembangan permintaan pasar akan komputasi kelas enterprise, saat ini ASUS mulai menapaki pasar server dengan lebih serius. Kali ini EFISON kedatangan server ASUS ESC4000A-E10 yang ditenagai CPU AMD EPYC Rome dengan arsitektur AMD Zen 2 dan GPU NVIDIA dengan arsitektur NVIDIA Turing.

ASUS ESC4000A-E10
ASUS ESC4000A-E10

Spesifikasi

Tipe Model/Spesifikasi
Processor / System Bus 1 x Socket SP3 (LGA 4094)
AMD EPYC™ 7002 Series
Memory Total Slots : 8 (8-channel)
Capacity : Maximum up to 2048GB RDIMM
Memory Type :
DDR4 3200 RDIMM
DDR4 3200 LRDIMM
DDR4 3200 LR-DIMM 3DS
Memory Size :
256GB, 128GB, 64GB, 32GB, 16GB
* Refer to support page for more information
Expansion Slots Rear:
– 4 x PCIe x16 slots (Gen4 x16 link, FH,FL) for dual-slot GPU cards or 8 x PCIe x16 slots (Gen4 x8 link, FH,FL) for single-slot GPU cards
– 2 x PCIe x16 slots (Gen4 x16 link, LP,HL)

Front:
– SKU-1 (default)
1 x PCIe x8 slot (Gen4 x8 link, LP,HL)

– SKU-2 (by request)
1 x PCIe x8 slot (Gen4 x8 link, LP,HL) or
1 x OCP3.0 slot (Gen4 x8 link) by switching cables

– SKU-3 (by request)
1 x PCIe x8 slot (Gen4 x8 link, LP,HL) or
2 x M.2 socket (Gen4 x4 link, up to 22110 module) by switching cables
Storage SATA Controller :
CPU Integrated
1 x M.2 connector(2242/2260/2280/22110) PCIe mode (PCIe Gen4 x4 link 22110/2280/2260)
Optional kits Controller :
ASUS PIKE II 3008-8i 8-port SAS 12G RAID card
ASUS PIKE II 3108-8i 8-port SAS 12G HW RAID card
Drive Bays 8 x 3.5″ or 2.5″ Hot-swap Storage Bays
(Backplane Supports 4 x SATA/SAS + 4 x SATA/SAS/NVMe Devices)

*default setting supports 2 x NVMe devices.
Networking 1 x Dual Port Intel I350-AM2 Gigabit LAN controller + 1 x Mgmt LAN
Graphic
Aspeed AST2500 with 64MB VRAM
Front I/O Ports 1 x Q-code/port-80 LED
4 x USB 3.2 Gen1 ports
Rear I/O Ports 2 x USB 3.2 Gen1 port
1 x VGA port
1 x Management port (RJ45)
2 x Gigabit LAN ports (RJ45)
Switch/LED Rear Switch/LED:
1 x Power switch/LED
1 x Location LED
1 x Message LED
1 x HDD Access LED

Front Switch/LED:
1 x Power switch/LED
1 x Location switch/LED
2 x LAN LED
1 x Message LED
1 x HDD LED
OS Support Please find the latest OS support from https://www.asus.com/event/Server/OS_support_list/OS.html
Management Solution ASUS Control Center
On-Board ASMB9-iKVM for KVM-over-IP
Regulatory Compliance BSMI, CE, RCM, FCC(Class A)
Dimensions 800mm x 440mm x 88.9mm (2U)
31.50″ x 17.22″ x 3.5″
Form Factor 2U
Power Supply 1+1 Redundant 1600W 80 PLUS Platinum Power Supply
Rating: 100-127Vac/200-240Vac,12.9A/9.5A,50-60Hz

1+1 Redundant 2200W 80 PLUS Platinum Power Supply
Rating: 100-127Vac/200-240Vac,14A/12.6A,47-63Hz
Environment Operation temperature: 10℃ ~ 35℃ / Non operation temperature: -40℃ ~ 70℃
Non operation humidity: 20% ~ 90% ( Non condensing)
Note
*Users need to remove Slimline cables from the PCIe riser board, and re-connect the cables to the backplane for total 4 x NVMe devices.

CPU yang datang ke lab EFISON untuk pengujian ASUS ESC4000A-E10 adalah AMD EPYC 7502P. EPYC 7502P memiliki 32 core dengan SMT yang menjadikannya 64 thread, boost clock hingga 3.35 GHz, dan L3 cache sebesar 128 MB. Dari sisi dukungan ia mendukung memori DDR4 ECC 8-channel dan PCIe 4.0 hingga 128-lanes. CPU ini merupakan salah satu dari jajaran CPU AMD EPYC Rome yang menggunakan mikroarsitektur Zen 2 dengan fabrikasi 7 nm TSMC.

AMD EPYC 7502P dengan memori 8-channel
AMD EPYC 7502P dengan memori 8-channel

Dari sisi GPU, EFISON juga diberi kesempatan untuk mencoba sebuah Nvidia Tesla T4. GPU ini menggunakan mikroarsitektur Turing dengan fabrikasi 12 nm. Tesla T4 menggunakan chip TU104 dengan 40 SM dan 2560 CUDA core. Ia memiliki ukuran single slot sehingga dapat dengan mudah dipasang di rack server 1U maupun 2U. Dari segi konsumsi daya ia hanya menggunakan daya dari slot PCIe yang tidak sampai 75 W. GPU ini dapat dipasang hingga 8 buah di server ASUS ESC4000A-E10 melalui slot PCIe di sebelah kanan dan kiri unit server.

Unit Nvidia Tesla T4
Unit Nvidia Tesla T4

8 slot PCIe x16 yang ada di sebelah kiri dan kanan
8 slot PCIe x16 yang ada di sebelah kiri dan kanan

ASUS ESC4000A-E10 mendukung hingga 8 buah 3.5″/2.5″ drive bay dengan konfigurasi maksimum 4 SATA/SAS + 4 SATA/SAS/NVMe. Pengguna juga dapat menambahkan RAID card PCIe sendiri di slot PCIe depan yang sudah disediakan.

8 buah drive bay
8 buah drive bay dengan dukungan hingga 4 NVMe

Slot tambahan untuk RAID card
Slot tambahan untuk RAID card


ASUS menyematkan server 2U ini dengan berbagai fitur untuk optimasi performa maupun fitur enterprise.

Performance Tuning

Pengguna dapat menyesuaikan pengaturan optimasi performa menyesuaikan workload secara otomatis berdasarkan profile dari ASUS. Untuk mengakses fitur Performance Tuning ini, pengguna cukup masuk ke System Setup (BIOS) dengan menekan DEL saat startup, lalu masuk ke tab menu Performance Tuning.

Performance Tuning
Performance Tuning

Pengguna dapat memilih setelan optimasi berdasarkan workload maupun benchmark.

Optimasi performa berdasarkan benchmark atau workload tertentu
Optimasi performa berdasarkan benchmark atau workload tertentu

Pada pengujian kali ini EFISON menggunakan opsi By Workload: HPC.

By Workload: HPC
By Workload: HPC

Pengguna juga dapat melakukan overclocking otomatis dengan enable menu overclocking di bawah. Pada pengujian kali ini, EFISON menggunakan setelan Overclocking: Disabled.

BMC WebUI

Pengguna dapat melakukan monitoring kondisi hardware, update firmware, update BIOS, maupun remote control melalui WebUI.

Antarmuka WebUI
Antarmuka WebUI


Pengujian performa dilakukan dengan berbagai software sintetis maupun software saintifik. Berikut adalah konfigurasi hardware dan software yang digunakan.

Tipe Model/spesifikasi
CPU 1 * AMD EPYC 7502P 32c/64t 180W
GPU 1 * Nvidia Tesla T4
RAM 128 GB DDR4-3200 ECC 8-channel
Storage 120 GB Intel SSD DC S3500 Series
OS CentOS 7.8
Kernel 5.9.1-1.el7.elrepo.x86_64

Berikut adalah software yang digunakan dalam pengujian. Seluruh software di-build dari source dengan berbagai optimasi compiler dan library untuk memaksimalkan performa dari software tersebut.

Software Versi Compiler/Library Optimasi
High-Performance LINPACK 2.3 Compiler:
GNU 9.3.0

MPI:
OpenMPI 4.0.4

BLAS:
Intel MKL 2020.0

 Compiler:
-march=znver2

BLAS:
MKL_DEBUG_CPU_TYPE=5
High-Performance Conjugate Gradient 3.1 Compiler:
GNU 9.3.0

MPI:
OpenMPI 4.0.4

(Khusus HPCG GPU, menggunakan binary dengan dukungan CUDA 11 dari web HPCG)

 Compiler:
-march=znver2
GROMACS 2020.3 Compiler:
GNU 9.3.0

MPI:
OpenMPI 4.0.4

BLAS:
BLIS AMD AOCL 2.2 (di-compile sendiri menggunakan GNU 10)

LAPACK:
LibFLAME AMD AOCL 2.2 (di-compile sendiri menggunakan GNU 10)

CUDA:
11.0

 Compiler:
-march=znver2

CUDA:
-arch=sm_75
NAMD 2.14 Compiler:
GNU 9.3.0

FFTW:
Intel MKL 2020.0

CUDA:
11.0

 Compiler:
-march=znver2

BLAS:
MKL_DEBUG_CPU_TYPE=5

CUDA:
-arch=sm_75

High-Performance LINPACK (HPL)

HPL adalah software yang menyelesaikan sistem linier padat acak dalam aritmatik double precision (64-bit) pada komputer dengan memori terdistribusi. HPL merupakan standar benchmark high-performance computing (HPC) dan superkomputer di dunia. Benchmark HPL juga digunakan sebagai tolok ukur performa 500 superkomputer tercepat di dunia yang dirangkum pada laman Top500.

Hasil pengujian HPL
Hasil pengujian HPL

Hasil pengujian HPL menghasilkan skor 1319.3 GFLOPS. Hasil yang dicatatkan oleh EPYC 7502P ini termasuk kencang apabila dibandingkan dengan berbagai CPU kelas workstation maupun enterprise. Sebagai perbandingan, berikut adalah hasil benchmark HPL oleh Dr. Donald Kinghorn dari Puget Systems terhadap berbagai CPU.

Hasil benchmark HPL dari Puget Systems
Hasil benchmark HPL pada berbagai CPU dari Puget Systems

High-Performance Conjugate Gradient (HPCG)

HPCG adalah software benchmark yang melakukan iterasi gradien konjugasi prakondisi multigrid menggunakan nilai floating-point double precision (64-bit). HPCG umum digunakan sebagai suplemen benchmark HPL dan menjadi ukuran efisiensi performa superkomputer dengan perhitungan (hasil HPCG dalam FLOPS/hasil HPL dalam FLOPS).

Pada pengujian HPCG, digunakan konfigurasi problem size 104 104 104 agar HPCG juga bekerja menguji memori dan tidak hanya berjalan di cache prosesor.

Konfigurasi hpcg.dat
Konfigurasi hpcg.dat

HPCG CPU

Hasil benchmark HPCG CPU
Hasil pengujian HPCG CPU

Pengujian HPCG CPU menghasilkan skor dari AMD EPYC 7502P sebesar 14.6655 GFLOPS. Sebagai pembanding, berikut adalah data hasil benchmark HPCG CPU dari Puget Systems.

Hasil benchmark HPCG pada berbagai CPU dari Puget Systems
Hasil benchmark HPCG pada berbagai CPU dari Puget Systems

Skor EPYC 7502P relatif lebih cepat dibanding Threadripper berkat keunggulan 8-channel memori dibanding Threadripper yang hanya 4-channel.

HPCG GPU

 

Hasil pengujian HPCG GPU
Hasil pengujian HPCG GPU

Pengujian HPCG GPU menghasilkan skor dari Nvidia Tesla T4 sebesar 43.4717 GFLOPS. Sebagai pembanding berikut adalah data hasil benchmark HPCG GPU dari Puget Systems.

Hasil benchmark HPCG pada berbagai GPU dari Puget Systems
Hasil benchmark HPCG pada berbagai GPU dari Puget Systems

Skor Nvidia Tesla T4 memang jauh lebih rendah apabila dibandingkan dengan GPU kelas konsumer high-end, namun perlu diingat pula Nvidia Tesla T4 memiliki ukuran yang lebih ringkas (hanya 1 slot PCIe) serta konsumsi daya maksimum yang lebih rendah. Hal ini akan sangat membantu apabila pengguna ingin memasang konfigurasi banyak GPU pada server ASUS ESC4000A-E10 ini.

GROMACS

GROMACS adalah software saintifik untuk melakukan perhitungan dinamika molekuler seperti mensimulasikan persamaan gerak Newton pada sistem dengan jutaan partikel. Ia didesain untuk molekul biokimia seperti protein, lipid, dan asam nukleat yang memiliki banyak interaksi ikatan kompleks.

Pengujian dilakukan menggunakan file input RNAse dodecahedron PME.

GROMACS CPU

GROMACS RNAse dodecahedron PME hanya menggunakan CPU
GROMACS RNAse dodecahedron PME hanya menggunakan CPU

Pada pengujian GROMACS RNAse menggunakan EPYC 7502P, didapatkan hasil hingga 130.709 ns/day. Task hanya menggunakan hingga 32 core untuk mempopulasi jumlah core fisik. Penggunaan hanya core fisik tanpa SMT menghasilkan performa yang lebih cepat dibandingkan dengan mempopulasi seluruh thread.

GROMACS CPU + GPU

GROMACS RNAse menggunakan CPU + GPU
GROMACS RNAse dodecahedron PME menggunakan CPU + GPU

Penggunaan GPU Nvidia Tesla T4 pada pengujian GROMACS RNAse dodecahedron PME menghasilkan akselerasi negatif. Hal ini dikarenakan jumlah GPU yang digunakan terlalu sedikit sehingga off-load tugas dari CPU ke GPU justru lebih tidak efisien. Terlihat bahwa wall time boros di Wait PME GPU gather dan Wait GPU NB local. Hasil pengujian mencatatkan hanya 86.739 ns/day, lebih kecil dibanding hanya menggunakan CPU.

NAMD

NAMD merupakan software saintifik dinamika molekuler paralel yang didesain untuk melakukan simulasi dari sistem biomolekuler besar. Software ini mampu scaling hingga ratusan core untuk simulasi biasa dan lebih dari 500.000 core untuk simulasi besar.

Pengujian dilakukan menggunakan file input ApoA1. Hasil ditunjukkan dalam bentuk skalabilitas terhadap jumlah core.

NAMD CPU

NAMD ApoA1 hanya CPU
NAMD ApoA1 hanya menggunakan CPU

Komputasi NAMD menggunakan AMD EPYC 7502P tanpa dibantu Nvidia Tesla T4 menghasilkan performa yang scaling hingga jumlah task NAMD sama dengan jumlah core fisik (32 core). Pada penggunaan 32 core, 7502P menghasilkan 9.69558 ns/day sedangkan penggunaan maksimum 64 task menghasilkan 10.2963 ns/day.

NAMD CPU + GPU

Hasil NAMD ApoA1 menggunakan CPU + GPU
Hasil NAMD ApoA1 menggunakan CPU + GPU

Komputasi NAMD menggunakan EPYC 7502P dengan dibantu Nvidia Tesla T4 menghasilkan performa komputasi yang jauh lebih baik, hingga lebih dari 32 ns/day.


Server ASUS ESC4000A-E10 ini merupakan server 2U menarik dengan berbagai fitur yang layak untuk digunakan dalam ekosistem HPC. Fitur optimasi performa menggunakan profile workload maupun software di BIOS, dukungan CPU AMD EPYC Rome dengan memori 8-channel DDR4-3200, serta slot PCIe 4.0 hingga 8 GPU menjadikan server ini sangat layak untuk dijadikan pilihan komputasi berat seperti saintifik, machine learning, AI inference, hingga workstation.

Terkait ketersediaan, ASUS menjanjikan server ini hadir di pasar Indonesia antara Q4 2020 atau Q1 2021. Mengenai prosesor AMD EPYC Milan (7003-series) yang akan datang kemungkinan akan didukung, tentunya hal ini menunggu pula informasi rilis dari AMD maupun ASUS.

The post Review ASUS ESC4000A-E10, server 2U dengan AMD EPYC dan Nvidia Tesla untuk HPC appeared first on Efison Lisan Teknologi.

]]>