Comments (25)
Thanks for the sharing!
Will look into this when finish the higher priority work in hand and update later.
from azurite.
@bluewe Apparently we needed to get the docker file download and include the node's ver 20.x explicitly. Ver 20.x is not by default selected otherwise.
from azurite.
@blueww Could you please chime in and let me know your thoughts on this issue? Thanks.
from azurite.
The above log you shared is from Storage SDK.
From the error I see: 1. the error happens on API BlobContainerClient.GetPropertiesInternal()
; 2. the error is client can't connect to 127.0.0.1:10000 successfully.
Would you please:
- Share the Azurite debug log when the failure happen? The debug log path is set with Azurite start parameter "-d /usr/local/lib/node_modules/azurite/debug.log" in your above Azurite start command. We need the Azurite debug log to do more investigation.
- Share the connection string you use (hide credential), or just share do you use http or https?
- Please check is there any config / process on your machine blocks the request send to 127.0.0.1:10000.
from azurite.
-
The debug log file was not created per the following screenshot, and it seems to be a new problem for you guys to solve:
-
The connection string is:
"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1;"
-
There is nothing blocking the request as the entire application used to work before upgrading the azurite and the nuget packages.
I also used --skipApiVersionCheck --loose
in the azurite cli but it was not conclusive.
from azurite.
@blueww any updates? Thanks.
from azurite.
@Arash-Sabet
We have never see any customer report azurite debug log not generated. So it’s almost not possible that there’s bug blocks azurite debug log generate.
Does the folder you show above on the docker , or on your client machine host docker? The debug log should be generated on the docker, so normally we will map the a folder on docker to a local folder, then generate debug log in the folder, then you can access debug log from local folder.
Another possible reason is azurite not start at all. There are something on your machine blocks azurite from starting.
from azurite.
@blueww The following failure message is all I have to share. This exception also happens when we do not use sudo
or with --loose
or --skipApiVersionCheck
switches.
sudo azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 -s -l /usr/local/lib/node_modules/azurite -d /usr/local/lib/node_modules/azurite/debug.log &
========================== Starting Command Output ===========================
/usr/bin/bash /azp/_work/_temp/5d5dabf2-5efa-43e2-974a-2ea32571caaf.sh
/usr/local/lib/node_modules/azurite/dist/src/common/persistence/MemoryExtentStore.js:53
return this._chunks.get(categoryName)?.chunks.get(id);
^
SyntaxError: Unexpected token '.'
at wrapSafe (internal/modules/cjs/loader.js:915:16)
at Module._compile (internal/modules/cjs/loader.js:963:27)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
at Module.load (internal/modules/cjs/loader.js:863:32)
at Function.Module._load (internal/modules/cjs/loader.js:708:14)
at Module.require (internal/modules/cjs/loader.js:887:19)
at require (internal/modules/cjs/helpers.js:74:18)
at Object. (/usr/local/lib/node_modules/azurite/dist/src/common/ConfigurationBase.js:7:29)
at Module._compile (internal/modules/cjs/loader.js:999:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
from azurite.
From the above error, I really have no idea.
And I can't repro this.
I will try to contact the code owner of "MemoryExtentStore.js" to see if any idea.
One workaround is: you might can switch back to 3.29.0, if you work doesn't depends on change of 3.30.0.
from azurite.
@blueww We have already had to downgrade the azurite to 3.29.0 and the NuGet packages too.
from azurite.
I get responds from the code owner of "MemoryExtentStore.js", however he seems also don't have much idea.
He mentioned:
I don't know. A syntax error may come from a very old version of node running on newer syntax .js file.
In memory persistence was introduced in 3.28.0. 3.29.0 works for the user but that also has in-memory persistence. Very confusing.
from azurite.
@blueww That sounds like an issue in the implementation of azurite. This is a hinderance to us and other people who use azurite within docker. The issue has to be diligently resolved by Microsoft as the owner of this product. The response from the code owner is not satisfying as he must nail down this issue and address it. With this issue around, upgrading azurite NuGet packages becomes impossible unless you guys come up with another workaround.
from azurite.
The problem is the current error log is not enough to get the root cause of the issue.
We can't repro this issue, and we don't get other customer report similar issue.
We will need either a repro environment or more error log/trace to investigation this issue.
Could you give the very detail steps to repro this issue?
Or share a repro environment?
If we could get or setup a repro environment, it would definitely help the investigation.
BTW, could you check the node version of your repro environment?
from azurite.
@blueww If other customers are not reporting this issue it does not mean that this problem does not exist. We cannot browse the entire internet to see what all people have to say or they may have not stumbled upon this issue yet. The error I shared is a fact from Azurite's error messages log that I collected.
I am unable to share an environment to reproduce this issue. But I hope the following docker file snippet leads your team to the issue:
FROM ubuntu:22.04
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends \
apt-transport-https \
apt-utils \
ca-certificates \
curl \
git \
iputils-ping \
jq \
lsb-release \
software-properties-common \
sudo \
zip \
unzip \
tar \
wget
RUN curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
RUN wget https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
RUN dpkg -i packages-microsoft-prod.deb
RUN rm packages-microsoft-prod.deb
RUN sudo apt-get update
RUN sudo apt-get install -y dotnet-sdk-8.0
RUN sudo apt-get -y install nodejs
RUN sudo apt-get install -y nodejs npm
RUN sudo npm install -g azurite
RUN sudo npm fund
RUN sudo mkdir azurite
There are other commands that I excluded from this docker file deemed unnecessary in this scope. That's all I can provide.
from azurite.
For we don't get other customer report similar issue, I don't mean the issue not exist, but mean we don't have repro way from other customer, so can only get the way to repro it from you. And no other customer meet this issue, might will make the issue not so high priority compare with other features/issues requested by more customers.
We might can try to repro this with you above script. However, could you give the detail step by step guild on how to use the script to setup the docker, it will definitely make the repro more efficiently.
from azurite.
@blueww I would give it a high priority to have it looked into.
To reproduce the issue: I'm sure there are other folks using k8s for their build agents. Please have your team follow the steps outlined per this blog post and include the script I shared when building the k8s's docker image. I guess that blog post is a re-iteration of a Microsoft's document/practice echoed by the author in the context of making a build agent.
Write a C# Xunit method to create a container on the Azurite instance, run it through a build pipeline, and you will observe the issue. This is how you'd normally launch Azurite inside the docker image from Azure DevOps' YAML file:
- task: Bash@3
displayName: Launching Azurite
inputs:
targetType: 'inline'
script: 'sudo azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 -s -l /usr/local/lib/node_modules/azurite -d /usr/local/lib/node_modules/azurite/debug.log &'
from azurite.
I can get very similar error as yours when use a old version node (10.24.1) as following.
Azurite has already deprecated node 10/12. see #1830
Please try again with a later node version if you have not. I would suggest using LTS version v20.15.0. https://nodejs.org/en
If you have already used a supported Node version, please indicate which node version you are using.
D:\code\azurite\dist\src\common\persistence\MemoryExtentStore.js:53
return this._chunks.get(categoryName)?.chunks.get(id);
^SyntaxError: Unexpected token .
at Module._compile (internal/modules/cjs/loader.js:723:23)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Object. (D:\code\azurite\dist\src\common\ConfigurationBase.js:7:29)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
from azurite.
@blueww The following lines are from the script I already shared in this issue. Don't they imply that the latest node version is utilized?
RUN sudo apt-get -y install nodejs
RUN sudo apt-get install -y nodejs npm
from azurite.
@blueww The following lines are from the script I already shared in this issue. Don't they imply that the latest node version is utilized?
RUN sudo apt-get -y install nodejs RUN sudo apt-get install -y nodejs npm
I am not sure.
A better way is get the node version after (or just before) the issue repro, and see if it use the correct node version.
from azurite.
@blueww I am confused. Can you copy/paste your script that worked to see how you resolved the issue?
from azurite.
@blueww I am confused. Can you copy/paste your script that worked to see how you resolved the issue?
You can just run "node --version" to get the node version.
from azurite.
@blueww that's not my point. My point is that what script you used to include that specific node version in your test script that led to success.
from azurite.
@blueww that's not my point. My point is that what script you used to include that specific node version in your test script that led to success.
For the setup script, this is out of Azurite scope, we might not be the best person to answer it. You might can contact related team like ADO team for how to setup the right node version.
Please make sure you are using right node version, and if so still any azurite issue happen, we can continue the investigation.
from azurite.
@blueww The azurite clearly has a problem and to me the following script is downloading the latest version of node.
RUN sudo apt-get -y install nodejs
RUN sudo apt-get install -y nodejs npm
For the setup script, this is out of Azurite scope, we might not be the best person to answer it. You might can contact related team like ADO team for how to setup the right node version.
The setup script leads you to reproduce the problem and I don't think you cannot keep it out of this scope. This is a perfect test to reveal the problem in azurite. I would engage the team to solve the problem as you had asked me before to guide you through the reproduction steps. This is going to be my last answer and post on this issue.
from azurite.
Thanks for the reply!
However, Azurite can only work on the right node version. But how to setup the right node version on customer's machine is out of Azurite scope, customer owners the setup of node.
The error of this issue indicate it's very possible that a wrong node version is using, so to continue the investigation, please get the node version on your repro machine, and see if it's the correct node version.
For the installation script, we are not expert of node installation, so not sure if any configuration on client machine can impact the real node version be installed. The better way to check is: run "node --version" to get the real node version after the issue repro on the same machine.
from azurite.
Related Issues (20)
- Azurite cannot handle encoded blob names HOT 5
- Cannot create a blob container from Azure Java client 12.26.0 HOT 3
- Cannot Create/Delete a table with Azurite and WindowsAzure.Storage SDK HOT 3
- XStore C++ tests are failing because Azurite returns wrong md5 HOT 16
- Azurite doesn't match Azure when downloading byte range from empty blob HOT 2
- SAS download returns 403 HOT 7
- Batch blob requests fail when multipart boundary contains '=' HOT 4
- Update mysql2 version HOT 5
- Metadata Vanishing HOT 12
- UserDelegationKey fails with 500 HOT 2
- Azurite HTTP Interface Impossible to Use from the Command Line. HOT 1
- BeginCopyAsync is throwing wrong exception when source blob is not found. HOT 1
- Upload from URL not working with a SAS URL HOT 1
- Not being able to launch Azurite from DockerHub. HOT 4
- The value for one of the HTTP headers is not in the correct format HOT 2
- Please Document Azurite Logging in VS CODE - not obvious how to see logs HOT 1
- Cannot add --skipApiVersionCheck when using testcontainers HOT 2
- Docker container is not up-to-date HOT 2
- Azurite returns 500 for all PUT requests after an unexpected client restart HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from azurite.