I've been able to append/create a table from a Pandas dataframe using the pandas-gbq package. In particular using the to_gbq method. However, When I want to check the table using the BigQuery web UI I see the following message:

This table has records in the streaming buffer that may not be visible in the preview.

I'm not the only one to ask, and it seems that there's no solution to this yet.

So my questions are:
1. Is there a solution to the above problem (namely the data not being visible in the web UI).
2. If there is no solution to (1), is there another way that I can append data to an existing table using the Python BigQuery API? (Note the documentation says that I can achieve this by running an asynchronous query and using writeDisposition=WRITE_APPEND but the link that it provides doesn't explain how to use it and I can't work it out).

That message is just a UI notice, it should not hold you back.

To check data run a simple query and see if it's there.

To read only the data that is still in Streaming Buffer use this query:

#standardSQL    
SELECT count(1) 
FROM `dataset.table` WHERE _PARTITIONTIME is null