tests/system/test_pandas.py::test_insert_rows_from_dataframe is flaky

[jimfulton]

In CI, this test has failed at least twice in the last day.

> assert len(row_tuples) == len(expected)
E AssertionError: assert 0 == 6
E + where 0 = len([])
E + and 6 = len([(1.11, True, 'my string', 10), (2.22, False, 'another string', 20), (3.33, False, 'another string', 30), (4.44, True, 'another string', 40), (5.55, False, 'another string', 50), (6.66, True, None, 60)])

tests/system/test_pandas.py:696: AssertionError

Perhaps this due to eventual consistency and we need to poll until we get 6 results.

[plamut]

Interestingly, I haven't observed this in the last 2+ years. As per docs, streamed data should be available within a few seconds, but something might have changed so that even this is sometimes too late.

@shollyman Is there a reasonable upper bound on those "few seconds"? Just to know how long does it make sense to keep polling in the test.

[plamut]

Also linking a comment by @tseaver for reference.

[tswast]

Queries should be reading from the streaming buffer, so I'm a bit surprised this started flaking.

Edit: That's why we run a query here instead of using list_rows

python-bigquery/tests/system/test_pandas.py

Lines 672 to 678 in 1246da8

	rows = list(
	bigquery_client.query(
	"SELECT * FROM `{}.{}.{}`".format(
	table.project, table.dataset_id, table.table_id
	)
	)
	)

[tswast]

I'm seeing the "within a few seconds" language now. I think that's a bit different from how it used to be. Polling until we get 6 results (max a few times) seems the right option.