Skip to content

feat(cambricon): enable TaskTopo graph capture - #1434

Open
gongchensu wants to merge 1 commit into
InfiniTensor:issue/1373from
gongchensu:fix/cambricon-tasktopo-capture-issue1373
Open

feat(cambricon): enable TaskTopo graph capture#1434
gongchensu wants to merge 1 commit into
InfiniTensor:issue/1373from
gongchensu:fix/cambricon-tasktopo-capture-issue1373

Conversation

@gongchensu

Copy link
Copy Markdown
Collaborator

Avoid queue synchronization while CNRT is capturing, while preserving the synchronous eager-execution contract outside capture.

Record descriptor-owned metadata with cnrtMemcpyAsync_V2 and pass temporary elementwise input pointers as kernel arguments so captured graphs can safely reuse frozen workspaces across batch shapes.

Initialize the Cambricon runtime and freeze cached allocations when they become visible to captured graphs so memory trimming cannot invalidate recorded pointers.

Avoid queue synchronization while CNRT is capturing, while preserving the synchronous eager-execution contract outside capture.

Record descriptor-owned metadata with cnrtMemcpyAsync_V2 and pass temporary elementwise input pointers as kernel arguments so captured graphs can safely reuse frozen workspaces across batch shapes.

Initialize the Cambricon runtime and freeze cached allocations when they become visible to captured graphs so memory trimming cannot invalidate recorded pointers.

@wooway777 wooway777 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果syncQueueIfNotCapturing意味着录图的时候不需要同步,那么不录图的时候是否也不需要同步呢?

@gongchensu

Copy link
Copy Markdown
Collaborator Author

如果syncQueueIfNotCapturing意味着录图的时候不需要同步,那么不录图的时候是否也不需要同步呢?

eager模式下异步调用依赖的handle、descriptor、workspace 或 host metadata 生命周期还是需要依赖同步来保证不出问题,最关键就是图的CNRT是会将capture期间提交的任务记录成tasktopo节点,graph会持续保存整套operator和planned_meta,graph可见的内存地址也会被冻结,因此是不需要用queue来同步,但是eager模式下,每个算子将任务提交到queue后就返回,workspace会被allocator标记为可复用,这些资源需要同步来保证生命周期。总之目前还是需要这样,后续如果想eager模式不用同步需要单独引入event延迟回收或per-stream resorce pool,并且重新管理CNNL handle、descriptor、metadata和workspace生命周期

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants