Possibility to use @db_session w/ few concurrent multiprocessing.Process

[surpr1ze]

When I start few processes, which are using the same @db_session, at the same time a get the following exception:

    record = get((row.id, row.user_id) for row in Record \
  File "<auto generated="" wrapper="" of="" get()="" function="">", line 2, in get
  File "/root/Projects/dns-stat-collector/.venv/lib/python2.7/site-packages/pony/utils.py", line 88, in cut_traceback
    return func(*args, **kwargs)
  File "/root/Projects/dns-stat-collector/.venv/lib/python2.7/site-packages/pony/orm/core.py", line 4260, in get
    return make_query(args, frame_depth=3).get()
  File "<auto generated="" wrapper="" of="" get()="" function="">", line 2, in get
  File "/root/Projects/dns-stat-collector/.venv/lib/python2.7/site-packages/pony/utils.py", line 88, in cut_traceback
    return func(*args, **kwargs)
  File "/root/Projects/dns-stat-collector/.venv/lib/python2.7/site-packages/pony/orm/core.py", line 4531, in get
    objects = query[:2]
  File "<auto generated="" wrapper="" of="" __getitem__()="" function="">", line 2, in __getitem__
  File "/root/Projects/dns-stat-collector/.venv/lib/python2.7/site-packages/pony/utils.py", line 88, in cut_traceback
    return func(*args, **kwargs)
  File "/root/Projects/dns-stat-collector/.venv/lib/python2.7/site-packages/pony/orm/core.py", line 4721, in __getitem__
    return query._fetch(range=(start, stop))
  File "/root/Projects/dns-stat-collector/.venv/lib/python2.7/site-packages/pony/orm/core.py", line 4433, in _fetch
    for sql_row in cursor.fetchall() ]
ProgrammingError: no results to fetch

Thanks!

[surpr1ze]

Need to disconnect() from db, before spawning new process

[kozlovsky]

I'm glad that db.disconnect() works for you. But I think that ideally Pony should be able to determine by itself that the stored connection was created in another process. This way Pony will ensure that the user will never experience any problem even if he doesn't know anything about possible multiprocessing issues.

In order to do this we need to store process id along with the connection, and then, when getting the connection from the pool, we need to compare stored process id with the current pid value and discard connection if the pid value differs.

[kozlovsky]

Now this problem should be completely solved. It is still recommended to call db.disconnect() before forking, but if a programmer forgets to do it, Pony should notice this and open a new connection instead of using the old one.

The only restriction remaining is that the programmer should not to do forking while a db_session is active, but it looks like a common sense.