Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON serialization error in admin view when using PickleEncoder #135

Open
huubbouma opened this issue Nov 13, 2022 · 2 comments
Open

JSON serialization error in admin view when using PickleEncoder #135

huubbouma opened this issue Nov 13, 2022 · 2 comments
Assignees

Comments

@huubbouma
Copy link

If you use the PickleEncoder for the serialization of the arguments to actors:

DRAMATIQ_ENCODER = "dramatiq.PickleEncoder"

Then an error will appear if you go to the task details in the admin interface:
TypeError: Object of type Application is not JSON serializable

huubbouma pushed a commit to huubbouma/django_dramatiq that referenced this issue Nov 13, 2022
…k view still works by showing a representation of the kwargs which is json serializable
huubbouma pushed a commit to huubbouma/django_dramatiq that referenced this issue Nov 13, 2022
…k view still works by showing a representation of the kwargs which is json serializable
@amureki
Copy link
Collaborator

amureki commented Nov 18, 2022

Hey @huubbouma, thanks for the bug report!

I am trying to reproduce this issue;
In my simple test, everything worked fine when I switched encoder:

CleanShot 2022-11-18 at 13 08 22

I guess, I am lacking your context.
May you share what your actor looks like? I see that type error complains about Application, what is it, Django model?

In general, according to best practices of working with queues (Dramatiq, Celery, regardless of which one) it is important to pass to actors as simple data (arguments) as possible:
https://dramatiq.io/best_practices.html#simple-messages

So, for example, instead of passing User object to change its email in a task, I'd recommend passing user_id, so the task can do a query to the database to access and change it. This way we are improving performance (as less data needs to go through the queue) and we are also protected from race conditions (if something will change User object while the task is in the queue, it would mean, task will operate with an outdated object and might overwrite new changes).

But please, correct me if my understanding of your context is wrong, I am happy to hear more to be able to reproduce it.

@amureki amureki self-assigned this Nov 18, 2022
@huubbouma
Copy link
Author

You're absolutely correct and we should use simple data, but my issue is that I'm migrating a hug codebase from another task queue and I don't have the the time now to convert existing code.
In my example we have django objects in the kwargs, which are not serializable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants