Exception: Cannot read weather station directory about meteostat-python HOT 9 CLOSED

Are you running Meteostat 0.3.0? Can you share the code snippet that you're using to query the stations?

from meteostat-python.

Here is the lines that concern the meteostat part. https://gist.github.com/gvangelatos/740b4a65434776ebc7a16aba410e191b
In the google colab meteostat is on the path /usr/local/lib/python3.6/dist-packages/meteostat-0.3.0.dist-info so i guess yes it is the version 0.3.0. Maybe the problem is with that i am using google colab?

from meteostat-python.

How are you setting the cache directory? I guess Google Colab doesn't allow the dot folders. Therefore, I would suggest changing the cache_dir to your custom path:

stations = Stations(lat = 37.983810, lon = 23.727539, cache_dir = '/root/meteostat/cache')

from meteostat-python.

i changed the directory with the line Stations._cache_dir = '/root/meteostat/cache'. But even though it created the folders there and a stations folder inside. It was empty and the same exception was thrown

from meteostat-python.

Ok, does it give you any other exceptions/details you can post here? It seems like it isn't able to store the file in that folder. I tested your code on my machine and it definitely works. Did it work on Google Colab before the latest version?

from meteostat-python.

Yes it worked before the last update of meteostat a few days ago. Here is all the Exceptions that are thrown: `---------------------------------------------------------------------------
ArrowTypeError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/meteostat/stations.py in init(self, lat, lon, radius, country, region, bounds, id, wmo, icao, daily, hourly, cache_dir, max_age, max_threads)
91 try:
---> 92 file = self._load(['stations/lib.csv.gz'])[0]
93 self._stations = pd.read_parquet(file['path'])

19 frames
/usr/local/lib/python3.6/dist-packages/meteostat/core.py in _load(self, paths)
107 for path in paths:
--> 108 files.append(self._download_file(path))
109

/usr/local/lib/python3.6/dist-packages/meteostat/core.py in _download_file(self, path)
89 # Save as Parquet
---> 90 df.to_parquet(local_path)
91

/usr/local/lib/python3.6/dist-packages/pandas/util/_decorators.py in wrapper(*args, **kwargs)
198 kwargs[new_arg_name] = new_arg_value
--> 199 return func(*args, **kwargs)
200

/usr/local/lib/python3.6/dist-packages/pandas/core/frame.py in to_parquet(self, path, engine, compression, index, partition_cols, **kwargs)
2371 partition_cols=partition_cols,
-> 2372 **kwargs,
2373 )

/usr/local/lib/python3.6/dist-packages/pandas/io/parquet.py in to_parquet(df, path, engine, compression, index, partition_cols, **kwargs)
275 partition_cols=partition_cols,
--> 276 **kwargs,
277 )

/usr/local/lib/python3.6/dist-packages/pandas/io/parquet.py in write(self, df, path, compression, index, partition_cols, **kwargs)
100
--> 101 table = self.api.Table.from_pandas(df, **from_pandas_kwargs)
102

/usr/local/lib/python3.6/dist-packages/pyarrow/table.pxi in pyarrow.lib.Table.from_pandas()

/usr/local/lib/python3.6/dist-packages/pyarrow/pandas_compat.py in dataframe_to_arrays(df, schema, preserve_index, nthreads, columns, safe)
500 arrays = list(executor.map(convert_column, columns_to_convert,
--> 501 convert_fields))
502

/usr/lib/python3.6/concurrent/futures/_base.py in result_iterator()
585 if timeout is None:
--> 586 yield fs.pop().result()
587 else:

/usr/lib/python3.6/concurrent/futures/_base.py in result(self, timeout)
431 elif self._state == FINISHED:
--> 432 return self.__get_result()
433 else:

/usr/lib/python3.6/concurrent/futures/_base.py in __get_result(self)
383 if self._exception:
--> 384 raise self._exception
385 else:

/usr/lib/python3.6/concurrent/futures/thread.py in run(self)
55 try:
---> 56 result = self.fn(*self.args, **self.kwargs)
57 except BaseException as exc:

/usr/local/lib/python3.6/dist-packages/pyarrow/pandas_compat.py in convert_column(col, field)
486 .format(col.name, col.dtype),)
--> 487 raise e
488 if not field_nullable and result.null_count > 0:

/usr/local/lib/python3.6/dist-packages/pyarrow/pandas_compat.py in convert_column(col, field)
480 try:
--> 481 result = pa.array(col, type=type_, from_pandas=True, safe=safe)
482 except (pa.ArrowInvalid,

/usr/local/lib/python3.6/dist-packages/pyarrow/array.pxi in pyarrow.lib.array()

/usr/local/lib/python3.6/dist-packages/pyarrow/array.pxi in pyarrow.lib._ndarray_to_array()

/usr/local/lib/python3.6/dist-packages/pyarrow/types.pxi in pyarrow.lib._datatype_to_pep3118()

/usr/local/lib/python3.6/dist-packages/pyarrow/error.pxi in pyarrow.lib.check_status()

ArrowTypeError: ('Did not pass numpy.dtype object', 'Conversion failed for column id with type string')

During handling of the above exception, another exception occurred:

Exception Traceback (most recent call last)
in ()
1 temperature_7daysback_df = []
2 #stations = Stations(lat = coordinates_df[cntry]["Lat"], lon = coordinates_df[cntry]["Long"])
----> 3 stations = Stations(lat = 37.983810,lon = 23.727539)
4 #stations = Stations(lat = 39.0742, lon = 21.8243)
5 station = stations.fetch(1)

/usr/local/lib/python3.6/dist-packages/meteostat/stations.py in init(self, lat, lon, radius, country, region, bounds, id, wmo, icao, daily, hourly, cache_dir, max_age, max_threads)
93 self._stations = pd.read_parquet(file['path'])
94 except:
---> 95 raise Exception('Cannot read weather station directory')
96
97 # Filter by identifier

Exception: Cannot read weather station directory`

from meteostat-python.

I guess the actual problem is:

ArrowTypeError: ('Did not pass numpy.dtype object', 'Conversion failed for column id with type string')

I would try to delete the cache folder and change all columns with type string to object. Place the following code before calling Stations():

Stations._types = {
    'id': 'object',
    'name': 'object',
    'country': 'object',
    'region': 'object',
    'wmo': 'object',
    'icao': 'object',
    'latitude': 'float64',
    'longitude': 'float64',
    'elevation': 'float64',
    'timezone': 'object'
}

It's important to remove the cache folder first and then run the code again.

from meteostat-python.

It worked!! Thank you very much you saved my thesis! You are amazing thanks for everything

from meteostat-python.

Thank you 😃
I'm glad it works now!

from meteostat-python.