Optimizing a CUDA Image-Processing Pipeline: From Bugs to Benchmarks
Public- 02 Sep, 2026

This practical CUDA example processes a stream of red, green, and blue images: data is first copied from the CPU to the GPU, converted from RGB to grayscale, then each 32-by-32-pixel tile is sorted so its median can be computed, and finally the medians are copied back to the host. The starting implementation defines two kernels: computeRGBToGray for the color conversion and computeMedian<TILE_WIDTH, HISTO_SIZE> for the per-tile median, while main `main` allocates memory, launches the kernels, and frees resources inside an OpenMP parallel loop over three images.
The median kernel has a subtle but serious flaw. Its shared-memory store tile[index] = d_image_gray[index] incorrectly uses a global index to write into shared memory, which is scoped to a single thread block. Running the binary produces an "illegal memory access" error, and compute-sanitizer pinpoints the exact cause: an out-of-bounds __shared__ write of one byte at line 55, reported for thread (0,3,0) in block (20,0,0).

To prevent such indexing mistakes, NVIDIA introduced a new launch API in CCCL. Kernels are launched with cuda::make_config, cuda::block_dims, cuda::grid_dims and cuda::launch, then the kernel accepts the configuration as its first parameter and uses cuda::gpu_thread.index(cuda::grid, config) and cuda::gpu_thread.index(cuda::block, config), to retrieve global and block indices separately. Raw pointers can also be replaced with cuda::std::span, cuda::std::mdspan and cuda::shared_memory_mdspan, which trigger assertions in debug mode when an out-of-bounds access occurs.

Once the code is bug-free, it is benchmarked with Nsight Systems. Instrumenting the code with NVTX ranges, such as nvtx3::scoped_range and nvtxRangePushA/nvtxRangePop - makes the timeline easier to read and shows that the median-computing kernel dominates the profile: it takes about 2.1 seconds per image and accounts for 98.5% of GPU time, while memory operations account for only 1.5%.

From the CPU side, the whole image-computation phase takes 6.8 seconds, with nearly all of that time going to the median kernels. This timeline gives developers a clear target for the next optimization steps: the median computation is by far the most expensive part of the pipeline, and the subsequent stages of the walkthrough focus on improving it.