OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

A solution to SQLAlchemy temporary table pain?

  • Thread starter Thread starter JamesHutchison
  • Start date Start date
J

JamesHutchison

Guest
It seems like the biggest drawback with SQLAlchemy is that it takes several steps backwards when it comes to working with temporary tables. A very common use case, for example, is to create a temporary table that is very specific to one task, throw some data in it, then join against it.

For starters, declaring a temporary table is verbose, and limited. Note that in this example I had to edit it because my classes actually inherit a base class, so what I give here may be slightly incorrect.

Code:
@as_declarative(metaclass=MetaBase)
class MyTempTable(object):

    __tablename__ = "temp"
    __table_args__ = {'prefixes': ['TEMPORARY']}

    id = Column(Integer(), primary_key=True)
    person_id = Column(BigInteger())
    a_string = Column(String(100))

Creating it is unintuitive:

Code:
MyTempTable.__table__.create(session.bind)

I also have to remember to explictly drop it unless I do something creative to get it to render with ON COMMIT DROP:

Code:
MyTempTable.__table__.drop(session.bind)

Also, what I just gave doesn't even work unless the temporary table is done "top level". I still haven't fully figured this out (for lack of wanting to spend time investigating why it doesn't work), but basically I tried creating a temp table in this manner inside of a nested transaction using session.begin_nested() and you end up with an error saying the relation does not exist. However, I have several cases where I create a temporary table inside of a nested transaction for unit testing purposes and they work just fine. Checking the echo output, it appears the difference is that one renders before the BEGIN statement, while the other renders after it. This is using Postgresql.

What does work inside of a nested transaction, and quite frankly saves you a bunch of time, is to just type out the damned sql and execute it using session.execute.

Code:
        session.execute(text(
            "CREATE TEMPORARY TABLE temp ("
            "  id SERIAL,"
            "  person_id BIGINT,"
            "  a_string TEXT"
            ") ON COMMIT DROP;"
        ))

Of course, if you do this, you still need a corresponding table model to make use of ORM functionality, or have to stick to using raw sql queries, which defeats the purpose of SQLAlchemy in the first place.

I'm wondering if maybe I'm missing something here or if someone has come up with a solution that is a bit more elegant.
<p>It seems like the biggest drawback with SQLAlchemy is that it takes several steps backwards when it comes to working with temporary tables. A very common use case, for example, is to create a temporary table that is very specific to one task, throw some data in it, then join against it.</p>

<p>For starters, declaring a temporary table is verbose, and limited. Note that in this example I had to edit it because my classes actually inherit a base class, so what I give here may be slightly incorrect.</p>

<pre><code>@as_declarative(metaclass=MetaBase)
class MyTempTable(object):

__tablename__ = "temp"
__table_args__ = {'prefixes': ['TEMPORARY']}

id = Column(Integer(), primary_key=True)
person_id = Column(BigInteger())
a_string = Column(String(100))
</code></pre>

<p>Creating it is unintuitive:</p>

<pre><code>MyTempTable.__table__.create(session.bind)
</code></pre>

<p>I also have to remember to explictly drop it unless I do something creative to get it to render with ON COMMIT DROP:</p>

<pre><code>MyTempTable.__table__.drop(session.bind)
</code></pre>

<p>Also, what I just gave doesn't even work unless the temporary table is done "top level". I still haven't fully figured this out (for lack of wanting to spend time investigating why it doesn't work), but basically I tried creating a temp table in this manner inside of a nested transaction using session.begin_nested() and you end up with an error saying the relation does not exist. However, I have several cases where I create a temporary table inside of a nested transaction for unit testing purposes and they work just fine. Checking the echo output, it appears the difference is that one renders before the BEGIN statement, while the other renders after it. This is using Postgresql.</p>

<p>What does work inside of a nested transaction, and quite frankly saves you a bunch of time, is to just type out the damned sql and execute it using session.execute. </p>

<pre><code> session.execute(text(
"CREATE TEMPORARY TABLE temp ("
" id SERIAL,"
" person_id BIGINT,"
" a_string TEXT"
") ON COMMIT DROP;"
))
</code></pre>

<p>Of course, if you do this, you still need a corresponding table model to make use of ORM functionality, or have to stick to using raw sql queries, which defeats the purpose of SQLAlchemy in the first place.</p>

<p>I'm wondering if maybe I'm missing something here or if someone has come up with a solution that is a bit more elegant.</p>
 

Latest posts

I
Replies
0
Views
1
Isaac P. Liu
I
U
Replies
0
Views
1
user3658366
U
G
Replies
0
Views
1
Giampaolo Levorato
G
M
Replies
0
Views
1
Marcelo Rodrigo Nascimento
M
Top