Your Question
i have written a Golang Import Script that uses a workerpool for concurrent usage, together with the great GORM as ORM Layer.
But everytime i run my code with more than 2 workers, at some point Gorm crashes with a segmentation fault.
Expected answer
Now i want to know if it is definitely my code ? or does anyone of you experience the same ?
Cheers
Adrian
Comment From: adnseek
panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1069701]
goroutine 48319 [running]: github.com/go-sql-driver/mysql.(mysqlStmt).writeExecutePacket(0xc0005a6018, {0xc00020c480, 0x24, 0x104d152?}) /Users/adriangier/go/pkg/mod/github.com/go-sql-driver/mysql@v1.7.0/packets.go:1102 +0x1d33 github.com/go-sql-driver/mysql.(mysqlStmt).Exec(0xc0005a6018, {0xc00020c480?, 0xc000028080?, 0x15520c8?}) /Users/adriangier/go/pkg/mod/github.com/go-sql-driver/mysql@v1.7.0/statement.go:58 +0xb0 github.com/go-sql-driver/mysql.(mysqlStmt).ExecContext(0xc0005a6018, {0x1551e98, 0xc000028080}, {0xc00023a600, 0x24, 0xc0005a6018?}) /Users/adriangier/go/pkg/mod/github.com/go-sql-driver/mysql@v1.7.0/connection.go:578 +0x165 database/sql.ctxDriverStmtExec({0x1551e98, 0xc000028080}, {0x15520c8, 0xc0005a6018}, {0xc00023a600?, 0x24, 0x24}) /usr/local/go/src/database/sql/ctxutil.go:65 +0xcc database/sql.resultFromStatement({0x1551e98, 0xc000028080}, {0x1551860, 0xc0004aa120}, 0xc000972df8, {0xc0005b0000, 0x24, 0x40}) /usr/local/go/src/database/sql/sql.go:2663 +0x145 database/sql.(DB).execDC(0x0?, {0x1551e98, 0xc000028080}, 0xc0000f4090, 0x0?, {0xc0005aa000, 0x273}, {0xc0005b0000, 0x24, 0x40}) /usr/local/go/src/database/sql/sql.go:1700 +0x473 database/sql.(Tx).ExecContext(0xc0015d0000, {0x1551e98, 0xc000028080}, {0xc0005aa000, 0x273}, {0xc0005b0000, 0x24, 0x40}) /usr/local/go/src/database/sql/sql.go:2491 +0xb7 gorm.io/gorm/callbacks.Create.func1(0xc00143bb90) /Users/adriangier/go/pkg/mod/gorm.io/gorm@v1.24.3/callbacks/create.go:96 +0x763 gorm.io/gorm.(processor).Execute(0xc0003f9cc0, 0x6?) /Users/adriangier/go/pkg/mod/gorm.io/gorm@v1.24.3/callbacks.go:130 +0x433 gorm.io/gorm.(DB).Create(0xc0009739c0?, {0x13fb700?, 0xc0001e2b40}) /Users/adriangier/go/pkg/mod/gorm.io/gorm@v1.24.3/finisher_api.go:24 +0xa5 main.importProduct({0xc0015c1380, 0x1a, 0x1a?}, {0x9, 0x14, 0x2, 0xd, 0x3, 0x19, 0xc, ...}, ...) /Users/adriangier/Sites/pipeline_import/main.go:554 +0x1230 main.importProducts.func1() /Users/adriangier/Sites/pipeline_import/main.go:365 +0x85 github.com/gammazero/workerpool.worker(0xc0014b8fd0?, 0x13032c6?, 0xc0009bc580?) /Users/adriangier/go/pkg/mod/github.com/gammazero/workerpool@v1.1.3/workerpool.go:237 +0x2a created by github.com/gammazero/workerpool.(WorkerPool).dispatch /Users/adriangier/go/pkg/mod/github.com/gammazero/workerpool@v1.1.3/workerpool.go:197 +0x2b8
Comment From: jinzhu
no，are you using one transaction for concurrent writes
Comment From: adnseek
Yes. I just use db.CreateIs this maybe my fault ?Von unterwegs gesendet. Am 24.01.2023 um 02:01 schrieb Jinzhu @.***>: no，are you using one transaction for concurrent writes
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>
Comment From: adnseek
Shall I follow this example ?
https://gorm.io/docs/transactions.html
Create a transaction per worker ?
Am 24.01.2023 um 06:57 schrieb Adrian @.***>:
Yes. I just use db.Create
Is this maybe my fault ?
Von unterwegs gesendet.
Am 24.01.2023 um 02:01 schrieb Jinzhu @.***>:
no，are you using one transaction for concurrent writes — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Comment From: adnseek
I tried to wrap the import in a transaction
At the end of the import (where db is the transaction I hand over with tx
I create the product
But I still receive
Any help would be awesome
Thank You Jinzhu
Adrian
Am 24.01.2023 um 09:14 schrieb Adrian Gier @.***>:
Shall I follow this example ?
https://gorm.io/docs/transactions.html
Create a transaction per worker ?
Am 24.01.2023 um 06:57 schrieb Adrian @.***>:
Yes. I just use db.Create
Is this maybe my fault ?
Von unterwegs gesendet.
Am 24.01.2023 um 02:01 schrieb Jinzhu @.***>:
no，are you using one transaction for concurrent writes — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Comment From: adnseek
Comment From: github-actions[bot]
This issue has been automatically marked as stale because it has been open 360 days with no activity. Remove stale label or comment or this will be closed in 180 days