Comments (36)
@spyamine I'll try your instructions and see if i can build a Windows Wheel, if I can, I'll upload it to PyPI so people will be able to just install without compiling
from arctic.
To further clarify @NickAltmann's points
You can get the unofficial windows lz4 binary from here: http://www.lfd.uci.edu/~gohlke/pythonlibs/#python-lz4
Download and install that via pip install C:/path/to/lz4_pip_wheel.whl
Then for the stdint header issue:
- Clone the artic github repository
- download this file: http://www.azillionmonkeys.com/qed/pstdint.h into the src/ folder in the local github directory you cloned
- rename the file to stdint.h
- run "pip install C:/path/to/github_repos/arctic"
Good to go! Thanks Nick
from arctic.
latest version compiled with VC++ 2015.3 v140 toolset for desktop (Visual Studio 2017 component):
arctic-1.64.0-cp36-cp36m-win_amd64.whl.zip
from arctic.
The easiest solution for stdint on python 2 is to use one of the free versions of these. I do this here
https://github.com/bashtage/randomgen/blob/master/randomgen/src/pcg32/pcg32.h
from arctic.
I just compiled it on Windows/Python2.7/x64 without issue from Master
https://github.com/bashtage/arctic/blob/py27-win/dist/arctic-1.64.0-cp27-cp27m-win_amd64.whl
from arctic.
Please ignore. I was able to compile and run the code, but had to manually copy all the modules within Arctic to the site-packages directory.
Many thanks!
from arctic.
Oh super! What did you have to do? We don't run this code on Windows here, so anything we can do to make this seamless would be good...
from arctic.
Here are my steps:
-
Install MinGW. Make sure to install the following packages:
Under Basic Setup:
mingw32-base, mingw32-gcc-g++
Under All Packages:
mingw32-pthreads-w32 -
Make sure the mingw bin directory is inside your PATH environment
-
Go into your Python27 directory then under Lib\distults\distutils.cfg, add the following:
[build]
compiler=mingw32
-
Do a pip install for numpy, pandas.
-
Finally do the install for arctic. Fingers crossed you won't get any error messages.
-
Create a simple python script that does an import to Arctic. If you get an error when running the code, which I did get, can try this solution: I had to directly copy the rest of the subdirectories with the .py files from the installation source directly into the site-packages/arctic folder myself. I am not sure if this is a bug with the installer or I am just doing it incorrectly.
Either way it seems to run now!
from arctic.
Would be great to get this automated so we can have some windows eggs / wheels to upload to PyPI
from arctic.
I am getting the same error message as beartastic ("fatal error C1083: Cannot open include file: 'stdint.h': No such file or directory". I followed his steps but it didn't help. Any idea? Thanks.
from arctic.
We don't technially require the native bits (they just provide parallel compression), you can run arctic without them as per:
#52 (comment)
from arctic.
I had two problems getting up and running on Windows.
The first was with the lz4 dependency. This was failing because the compile uses gcc specific command line arguments, causing the MSVC compile to fail. Not sure if I'm missing something, I've posed the question here.
The second problem was the missing stdint.h mentioned above, this was affecting both the lz4 install and arctic. This header is missing from VS 2008, the version on which the Python 2.7 compiler is based, it is there later versions. I fixed this locally by just grabbing a version someone has kindly produced, see here. To make the arctic code portable you'd need to define the missing types. The stackoverflow question I just linked to gives an example. Equally, if you look at the lz4 code on PyPi your can see that the file lz4.c is implemented entirely in types that is defines based on compiler version, see the typedefs for BYTE, U16, etc.
from arctic.
Hi,
I've created a wheel for arctic for python 2.7 64bit. I'll run you through how I've proceeded.
0- install lz4 from this link http://www.lfd.uci.edu/~gohlke/pythonlibs/#python-lz4
1- clone the repo arctic into a folder
2- add the stdint header to src/ folder from the link http://www.azillionmonkeys.com/qed/pstdint.h
3- using the method to create a wheel from this link : https://cowboyprogrammer.org/building-python-wheels-for-windows/ . I've installed some compilers and created some environments depending on the target environment I want to work with I had created the following bat program :
:: Required: git, Visual Studio 10.0 Express, Visual Studio 9 Express,
:: Windows 7 SDK (.NET 4), Windows 7 SDK (.NET 3.5), Anaconda/Miniconda
:::::::::::::::::::::::::::::::::::
:: User configurable stuff start ::
:::::::::::::::::::::::::::::::::::
:: These depends on where you installed Anaconda.
:: Python2 or Python3 doesn't matter but you need both 32bit and 64bit versions.
set ANACONDA_BASE=C:\Anaconda2
::set ANACONDA_BASE64=C:\Users\YOURUSERNAME\Anaconda-64
:: Location of your package, can be a directory or a git repo.
:: A directory where the git repo is.
set PKG_REPO=C:\Users\Amine\Desktop\ArcticGit\arctic
:: A git repo
:: set PKG_REPO=git+https://github.com/CamDavidsonPilon/lifelines.git
:::::::::::::::::::::::::::::::::
:: User configurable stuff end ::
:::::::::::::::::::::::::::::::::
:: Save original path so we can restore it
set BASEPATH=%PATH%
:: Set up your conda environments like this, do it for both 32bit and 64bit.
:: navigate to the Anaconda\Scripts / Anaconda-64\Scripts directory to avoid setting any Paths
:: conda create -n py3.4 python=3.4 numpy pip mingw
:: conda create -n py3.3 python=3.3 numpy pip mingw
:: conda create -n py2.7 python=2.7 numpy pip mingw
:: These depend on what you named your environments above
set ENV27=envs\py2.7
::set ENV33=envs\py3.3
::set ENV34=envs\py3.4
:: Tell Python to select correct SDK version
set DISTUTILS_USE_SDK=1
set MSSdk=1
:: Set 32-bit environment
call "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x86
:: Python 2.7 32bit
:: Python2 requires Visual Studio 2008 compiler present (Express version is fine)
call "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat"
set CPATH=%PATH%
set PATH=%ANACONDA_BASE%\%ENV27%;%ANACONDA_BASE%\%ENV27%\Scripts;%CPATH%
pip install wheel -q
pip wheel --no-deps %PKG_REPO%
:: Set 64-bit environment
call "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x64
:: Python 2.7 64bit
:: 64-bit cl.exe is here
set CPATH=C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\x86_amd64;%CPATH%
set PATH=%ANACONDA_BASE64%\%ENV27%;%ANACONDA_BASE64%\%ENV27%\Scripts;%CPATH%
pip install wheel -q
pip wheel --no-deps %PKG_REPO%
Then I have the wheel that I've tested on a new environment where i did install lz4 for compression.
I hope it will work for you.
I'm happy to help if any problem
arctic-1.26.0-cp27-cp27m-win_amd64.whl.zip
from arctic.
Thanks @spyamine, this was convenient and worked for me - needed to uncomment the line conda create -n py2.7 python=2.7 numpy pip mingw
from arctic.
Happy to hear that!! I've comment it because I've already created the conda environment.
How can we get this conveniently automated ? I'm new to github by the way
from arctic.
another thing missing is timezone support in windows, see issue #43
from arctic.
@Yoshi9143's instructions worked for me!
from arctic.
@flaviodrt ok, I'll try his instructions and see if they work for me. Last time I tried, it did not work :(
from arctic.
Setup for Anaconda 3 (Windows, Python 3.5, 64-bit):
-
Install Visual C++ 2015 Build Tools (~4Gb):
1.1 Download: http://landinghub.visualstudio.com/visual-cpp-build-tools
1.2 Run visualcppbuildtools_full.exe -
Open Anaconda prompt
-
mkdir C:\PackageTmp
-
cd C:\PackageTmp
-
git clone https://github.com/manahl/arctic.git
-
cd C:\PackageTmp\arctic
-
python setup.py build --compiler=msvc
-
pip install wheel
-
pip wheel --no-deps C:\PackageTmp\arctic
-
Bingo: C:\PackageTmp\arctic\arctic-1.50.0-cp35-cp35m-win_amd64.whl
-
pip install arctic-1.50.0-cp35-cp35m-win_amd64.whl
arctic-1.50.0-cp35-cp35m-win_amd64.whl.zip
Best regards,
Vladimir Malkhanov
from arctic.
@buhbuhtig thanks I'll try this today and upload a wheel if I can get it to work!
from arctic.
@bmoscon If you have failed to install using the above instructions, please, check {Anaconda3 path}\Lib\distutils\cygwinccompiler.py file.
Before my successful experiment I changed the following part of code from:
def get_msvcr():
"""Include the appropriate MSVC runtime library if Python was built
with MSVC 7.0 or later.
"""
msc_pos = sys.version.find('MSC v.')
if msc_pos != -1:
msc_ver = sys.version[msc_pos+6:msc_pos+10]
if msc_ver == '1300':
# MSVC 7.0
return ['msvcr70']
elif msc_ver == '1310':
# MSVC 7.1
return ['msvcr71']
elif msc_ver == '1400':
# VS2005 / MSVC 8.0
return ['msvcr80']
elif msc_ver == '1500':
# VS2008 / MSVC 9.0
return ['msvcr90']
elif msc_ver == '1600':
# VS2010 / MSVC 10.0
return ['msvcr100']
else:
raise ValueError("Unknown MS Compiler version %s " % msc_ver)
into:
def get_msvcr():
"""Include the appropriate MSVC runtime library if Python was built
with MSVC 7.0 or later.
"""
msc_pos = sys.version.find('MSC v.')
if msc_pos != -1:
msc_ver = sys.version[msc_pos+6:msc_pos+10]
if msc_ver == '1300':
# MSVC 7.0
return ['msvcr70']
elif msc_ver == '1310':
# MSVC 7.1
return ['msvcr71']
elif msc_ver == '1400':
# VS2005 / MSVC 8.0
return ['msvcr80']
elif msc_ver == '1500':
# VS2008 / MSVC 9.0
return ['msvcr90']
elif msc_ver == '1600':
# VS2010 / MSVC 10.0
return ['msvcr100']
elif msc_ver == '1700':
# Visual Studio 2012 / Visual C++ 11.0
return ['msvcr110']
elif msc_ver == '1800':
# Visual Studio 2013 / Visual C++ 12.0
return ['msvcr120']
elif msc_ver == '1900':
# Visual Studio 2015 / Visual C++ 14.0
return ['vcruntime140']
else:
raise ValueError("Unknown MS Compiler version %s " % msc_ver)
Src: http://c-sharp-snippets.blogspot.de/2016/08/compiling-aspell-python.html
from arctic.
Hi, thanks @buhbuhtig can you please build a version for python 2.7 ?
from arctic.
Does your version support compression or not ?? Also how can we deactivate compression please ??
from arctic.
I don't think you can turn off compression.
from arctic.
@spyamine Sorry, I don't use 2.7 version.
from arctic.
$ pip install arctic-1.50.0-cp35-cp35m-win_amd64.whl
arctic-1.50.0-cp35-cp35m-win_amd64.whl is not a supported wheel on this platform.
Am I missing something..?
I'm using Python 3.6.2 :: Anaconda custom (64-bit) on Win 8 / 64 bit using Cygwin
from arctic.
from arctic.
Still getting the error
$ pip install wheel
Requirement already satisfied: wheel in c:\anaconda3\lib\site-packages
$ pip install arctic-1.50.0-cp35-cp35m-win_amd64.whl
arctic-1.50.0-cp35-cp35m-win_amd64.whl is not a supported wheel on this platform.
from arctic.
latest version compiled in windows 10
arctic-1.55.0-cp36-cp36m-win_amd64.whl.zip
from arctic.
1.55 and 1.56 compile and work fine on Windows.
from arctic.
Hi can you please compile fir python 2.7 or indicate how to do it?
from arctic.
Cool @bashtage Thank you for sharing.
You did not have any problem with the lz4 library for using compression ?
from arctic.
No problems. I use
https://www.microsoft.com/en-gb/download/details.aspx?id=44266
for compiling.
from arctic.
@bashtage than you for the PR - are you happy to close this ticket now?
from arctic.
AFAICT it builds correctly on Python 2.7, 3.5 and 3.6 Windows/x86_64 which should be enough to close this. It may not work on Win32 but I don't think it makes sense to put effort into 32 bit Windows.
from arctic.
i tested this on windows. All I did was install python 3.6 and the visual studio compiler toolchain and it built without issue (on windows 10).
from arctic.
Related Issues (20)
- Impossible to asignate datetime index
- Migrating existing tickstore to ArcticDB HOT 3
- Update tests to use MongoDB 4.4
- most recent version not pip installing on mac M1 HOT 3
- [Question] - how to design data (store_type/chunk_size) HOT 6
- MongoDB 4.2 EOL April 2023 - What's Next? HOT 8
- Dask integration for tickstore
- arctic.exceptions.QuotaExceededException: Mongo Quota Exceeded: xxx 10.366 / 10 GB used HOT 1
- Index Monotonic Sort Bug in class DateChunker
- best practice usage HOT 1
- Missing last chunk in CHUNK_STORE HOT 1
- argument of type 'NoneType' is not iterable (when updating) HOT 1
- When to upgrade to python3.10? HOT 1
- circleci build container new Unix version breaks mongo install HOT 1
- Test arctic with mongodb 4.4 HOT 1
- VersionStore delete old snapshots very slow with large numbers of snapshots HOT 1
- Versionstore: Arctic automatically assigns index a name - Perhaps needs some warning HOT 1
- Java interface (JDBC) support HOT 1
- using the numpy no more than 1.18.5. This will cause issue when work with other libraries, for example matplotlib. HOT 1
- "AttributeError: 'NoneType' object has no attribute 'append'" using 4 or more threads. Fewer threads works fine. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from arctic.