Use text output from function as new request
Continuing from the previous case , helped by @Erwin Brandstetter and @Craig Ringer, I have corrected my code to be the following. Note that my function is myresult()
now outputting text
, not the table (as in the first case, it makes no sense to output the table object, since we will need to define all of its columns in front, which basically defies the whole purpose):
CREATE OR REPLACE FUNCTION myresult(mytable text, myprefix text)
RETURNS text AS
$func$
DECLARE
myoneliner text;
BEGIN
SELECT INTO myoneliner
'SELECT '
|| string_agg(quote_ident(column_name::text), ',' ORDER BY column_name)
|| ' FROM ' || quote_ident(mytable)
FROM information_schema.columns
WHERE table_name = mytable
AND column_name LIKE myprefix||'%'
AND table_schema = 'public'; -- schema name; might be another param
RAISE NOTICE 'My additional text: %', myoneliner;
RETURN myoneliner;
END
$func$ LANGUAGE plpgsql;
Call:
select myresult('dkj_p_k27ac','enri');
When doing the above procedure, I am getting a text string which is basically a request. I'll refer to it below as "oneliner-output", just for the sake of simplicity.
"Oneline-output" looks like this (I just copy / paste it from one output cell that I have here):
"SELECT enrich_d_dkj_p_k27ac,enrich_lr_dkj_p_k27ac,enrich_r_dkj_p_k27ac FROM dkj_p_k27ac"
- Note that double quotes on either side of the statement were part of the output
myresult()
. I have not added them.
Now I understand the much more problematic idea of thinking about building a single function that would create "oneliner-output" and execute it. I can copy / paste "oneliner-output" into a new Postgres query window and execute it like a normal query, just fine, getting the columns and rows I want in the Data Output window.
I would like, however, to automate this step to avoid the copy / paste action. Is there a way in Postgres to use the output text
('oneliner-output') that I receive from a function myresult()
and execute it? Is it possible to create a second function that will receive the output myresult()
and use it to execute the query?
Along those lines, although I know the following script (here below) works and actually outputs exactly the columns and rows you want:
-- DEALLOCATE stmt1; -- use this line after the first time 'stmt1' was created
prepare stmt1 as SELECT enrich_d_dkj_p_k27ac,enrich_lr_dkj_p_k27ac,enrich_r_dkj_p_k27ac FROM dkj_p_k27ac;
execute stmt1;
-
I was thinking maybe something like the following scenario could potentially work, once configured correctly? I don’t know how to do it.
prepare stmt1 as THE_OUTPUT_OF_myresult(); execute stmt1;
Trying with refcursor
CREATE OR REPLACE FUNCTION show_mytable(ref refcursor) RETURNS refcursor AS $$
BEGIN
OPEN ref FOR SELECT enrich_d_dkj_p_k27ac,enrich_lr_dkj_p_k27ac,enrich_r_dkj_p_k27ac FROM dkj_p_k27ac; -- Open a cursor
RETURN ref; -- Return the cursor to the caller
END;
$$ LANGUAGE plpgsql;
Call:
BEGIN;
SELECT show_mytable('roy');
FETCH ALL IN "roy";
This procedure actually works and splits out the columns and rows you want, and again, I have to provide an exact SELECT statement.
Basically I would like to be able to and provide it as the output of my function instead myresult()
. Something like that:
CREATE OR REPLACE FUNCTION show_mytable(ref refcursor) RETURNS refcursor AS $$
BEGIN
OPEN ref FOR myresult(); -- Open a cursor
RETURN ref; -- Return the cursor to the caller
END;
$$ LANGUAGE plpgsql;
Call:
BEGIN;
SELECT show_mytable('roy');
FETCH ALL IN "roy";
source to share
The c trick PREPARE
doesn't work as it doesn't accept * text string * (value) like CREATE FUNCTION
, but a valid statement (code).
To convert data to executable code, you need to use dynamic SQL, i.e. EXECUTE
in the plpgsql function or DO
. This works without issue as long as the return type does not depend on the result of the first function myresult()
. You also came back to catch 22 as stated in my previous answer:
The key part is to declare the return type (string type in this case) in some way. You can create TABLE
, TEMP TABLE
or TYPE
for this purpose. Or you can use prepared statement or refcursor.
A prepared statement solution
You were very close. The downside to the puzzle piece is preparing the generated query with dynamic SQL .
Function for dynamically presenting an instruction
Create this function once . This is an optimized and safe version of your function myresult()
:
CREATE OR REPLACE FUNCTION f_prep_query (_tbl regclass, _prefix text)
RETURNS void AS
$func$
BEGIN
IF EXISTS (SELECT 1 FROM pg_prepared_statements WHERE name = 'stmt_dyn') THEN
DEALLOCATE stmt_dyn;
END IF; -- you my or may not need this safety check
EXECUTE (
SELECT 'PREPARE stmt_dyn AS SELECT '
|| string_agg(quote_ident(attname), ',' ORDER BY attname)
|| ' FROM ' || _tbl
FROM pg_catalog.pg_attribute
WHERE attrelid = _tbl
AND attname LIKE _prefix || '%'
AND attnum > 0
AND NOT attisdropped
);
END
$func$ LANGUAGE plpgsql;
I am using regclass
for the table name parameter _tbl
to make it unambiguous and safe with SQLi . Details:
The information schema does not include the oid column of system catalogs, so I switched to pg_catalog.pg_attribute
instead information_schema.columns
. It's faster too. There are pros and cons for this:
If a prepared statement with a name stmt_dyn
already exists, PREPARE
throws an exception. If applicable, remove the system view check pg_prepared_statements
and the following DEALLOCATE
.
More sophisticated algorithms allow you to manage multiple prepared statements per session, or take the name of the prepared statement as an optional parameter, or even use the hash55 of the query string as the name, but that's beyond the scope of this question.
Be aware that PREPARE
out of bounds transactions, once it PREPARE
succeeds, the prepared statement exists for the lifetime of the session. If the transfer transaction is aborted, PREPARE
does not change. ROLLBACK
cannot delete prepared statements.
Executing a dynamic query
Two requests, but only one server call. And very effective too.
SELECT f_prep_query('tbl'::regclass, 'pre'::text);
EXECUTE stmt_dyn;
Simpler and more efficient for most simple use cases than creating a temporary table or cursor and selecting / fetching from that (which would be the other options).
source to share
I also found a solution using refcursor .
I would be very glad if you could go through this, check and tell me if you think this is Kosher. To be honest, I'm not too sure what I came up with here as I'm not familiar with the syntax. But I was able to synthesize this using different examples I found on the internet. This seems to work for me. I would be very happy if you could formulate this solution for me and for other users - and let us know what you think about it.
First, let's create a function that builds a dynamic statement SELECT
:
CREATE OR REPLACE FUNCTION myresult2()
RETURNS text AS
$func$
DECLARE
myoneliner text;
mytable text := 'dkj_p_k27ac';
myprefix text := 'enri';
BEGIN
SELECT INTO myoneliner
'SELECT '
|| string_agg(quote_ident(column_name::text), ',' ORDER BY column_name)
|| ' FROM ' || quote_ident(mytable)
FROM information_schema.columns
WHERE table_name = mytable
AND column_name LIKE myprefix||'%'
AND table_schema = 'public'; -- schema name; might be another param
-- RAISE NOTICE 'My additional text: %', myoneliner; -- for debugging
RETURN myoneliner;
END
$func$ LANGUAGE plpgsql;
Now, let's create a second function that can output the TEXT line of the first function myresult2()
:
CREATE OR REPLACE FUNCTION show_mytable(ref refcursor)
RETURNS refcursor AS
$func$
DECLARE
mydynamicstatment text := myresult2();
BEGIN
OPEN ref FOR EXECUTE mydynamicstatment;
RETURN ref; -- return cursor to the caller
END;
$func$ LANGUAGE plpgsql;
Call:
BEGIN;
SELECT show_mytable('roy');
FETCH ALL IN "roy";
source to share