Export large query results to CSV

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Export large query results to CSV

cmbendre
Hi,

Some of our queries on Phoenix cluster gives millions of rows as a result. How do i export these results to a csv file or s3 location ?

The query output size is approximately in range of GBs

Thanks,
Chaitanya
Reply | Threaded
Open this post in threaded view
|

Re: Export large query results to CSV

Josh Elser-2
I am not aware of any mechanisms in Phoenix that will automatically
write formatted data, locally or remotely. This will require you to
write some code.

cmbendre wrote:

> Hi,
>
> Some of our queries on Phoenix cluster gives millions of rows as a result.
> How do i export these results to a csv file or s3 location ?
>
> The query output size is approximately in range of GBs
>
> Thanks,
> Chaitanya
>
>
>
> --
> View this message in context: http://apache-phoenix-user-list.1124778.n5.nabble.com/Export-large-query-results-to-CSV-tp3530.html
> Sent from the Apache Phoenix User List mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Export large query results to CSV

Mujtaba Chohan
You might be able to use sqlline to export. Use !outputformat csv and !record commands to export as CSV locally.

On Sun, May 14, 2017 at 8:22 PM, Josh Elser <[hidden email]> wrote:
I am not aware of any mechanisms in Phoenix that will automatically write formatted data, locally or remotely. This will require you to write some code.


cmbendre wrote:
Hi,

Some of our queries on Phoenix cluster gives millions of rows as a result.
How do i export these results to a csv file or s3 location ?

The query output size is approximately in range of GBs

Thanks,
Chaitanya



--
View this message in context: http://apache-phoenix-user-list.1124778.n5.nabble.com/Export-large-query-results-to-CSV-tp3530.html
Sent from the Apache Phoenix User List mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Export large query results to CSV

cmbendre
In reply to this post by Josh Elser-2
Thank you for the info.

I am open to writing code or contributing to the project if this feature is missing. Let me create an issue for this.

Reply | Threaded
Open this post in threaded view
|

Re: Export large query results to CSV

iec2011007
This post has NOT been accepted by the mailing list yet.
In reply to this post by Mujtaba Chohan
But this will only hold data which a single machine can handle ? Am i wrong