guitarrapc / git-shallow-clone-orb Goto Github PK

View Code? Open in Web Editor NEW

21.0 3.0 11.0 157 KB

circleci orb to execute git shallow clone for faster checkout.

Home Page: https://circleci.com/orbs/registry/orb/guitarrapc/git-shallow-clone

License: MIT License

circleci circleci-orbs git clone-repos

git-shallow-clone-orb's Introduction

Git-shallow-clone-orb

Usage

See the orb registry listing for usage guidelines.

Contributing

We welcome issues to and pull requests against this repository!

Publish

To publish orb, push new tag to remote.

Add Test

Add test job in .circleci/config.yml.

jobs:
  # Define one or more jobs which will utilize your orb's commands and parameters to validate your changes.
  integration-test-checkout:
    docker:
      - image: cimg/base:stable
    steps:
      - git-shallow-clone/checkout

Call it from integration-test_deploy job, and add as orb-tools/dev-promote-prod-from-commit-subject required job.

  integration-test_deploy:
    when: << pipeline.parameters.run-integration-tests >>
    jobs:
      - integration-test-checkout # <-- this line!

      - orb-tools/dev-promote-prod-from-commit-subject:
          orb-name: guitarrapc/git-shallow-clone
          add-pr-comment: false
          fail-if-semver-not-indicated: true
          publish-version-tag: false
          requires:
            - integration-test-checkout # <-- this line!
          filters:
            branches:
              only:
                - master
                - main

Basic orb setup

setup orb account and namespace.

# require perconal api tokens
$ circleci setup
$ circleci namespace create guitarrapc github guitarrapc
$ circleci orb create guitarrapc/git-shallow-clone

validate before publish.

$ cd ./src
$ circleci orb validate orb.yml

publish orb to the alpha.

$ cd ./src
$ circleci orb publish orb.yml guitarrapc/git-shallow-clone@dev:alpha

publish orb to the dev.

$ cd ./src
$ circleci orb publish orb.yml guitarrapc/git-shallow-clone@dev:0.x.0

publish orb to the production.

$ cd ./src
$ circleci orb publish promote guitarrapc/[email protected]

git-shallow-clone-orb's People

Contributors

Stargazers

Watchers

Forkers

digit-co tartale ankushg pi-george pranshusharma1 stevesta r-plus jprice-da15252 harlemsquirrel hotelengine chrisopperwall-qz

git-shallow-clone-orb's Issues

Doesn't work with submodules

Using regular CircleCI's -checkout call I can later fetch submodules without issue. However I'm unable to do that with this orbs shallow checkout.

Not sure what to give you in terms of code as my config file is rather large and I have no minimal example.

Setting git config to use ssh instead of https breaks cloning from public repositories

This line in git clone shallow:

git config --global url."ssh://[email protected]".insteadOf "https://github.com" || true

Is causing composer install to fail with:

[RuntimeException]                                                                                           
  Failed to clone https://github.com/Pi-George/laminas-file.git via https, ssh protocols, aborting.            
                                                                                                               
  - https://github.com/Pi-George/laminas-file.git                                                              
    Cloning into bare repository '/root/.cache/composer/vcs/https---github.com-Pi-George-laminas-file.git'...  
    [email protected]: Permission denied (publickey).                                                             
    fatal: Could not read from remote repository.                                                              
                                                                                                               
    Please make sure you have the correct access rights                                                        
    and the repository exists.                                                                                 
                                                                                                               
  - [email protected]:Pi-George/laminas-file.git                                                                  
    Cloning into bare repository '/root/.cache/composer/vcs/https---github.com-Pi-George-laminas-file.git'...  
    [email protected]: Permission denied (publickey).                                                             
    fatal: Could not read from remote repository.                                                              
                                                                                                               
    Please make sure you have the correct access rights                                                        
    and the repository exists.

Also worth noting, checkout is checking out a bitbucket repo, not a github repo.

feat: add auto restore base64 encoded id_rsa to .ssh/id_rsa

add functionality describe at #15 (comment).

A. Users who want restore $CHECKOUT_KEY to .ssh/id_rsa, you should set base64 id_rsa on environment variables, then git-shallow-clone-orb/checkout will decode it to .ssh/id_rsa". Doesn't work with submodules #15 (comment) (I need prepare this)

Need 2 function, "detect env var BASE64_CHECKOUT_KEY is exists" and "decode base64 to .ssh/id_rsa".

shallow clone failed with "ssh-keyscan -H bitbucket.org >> ~/.ssh/known_hosts"

How often

1/100? or 1/1000

Failed log

# github.com:22 SSH-2.0-babeld-3acafefe
# github.com:22 SSH-2.0-babeld-3acafefe
# github.com:22 SSH-2.0-babeld-3acafefe
# bitbucket.org:22 SSH-2.0-conker_e350bda95a app-191
# bitbucket.org:22 SSH-2.0-conker_e350bda95a app-130
Exited with code 1

Is there an existing issue for this?

I have searched the existing issues

Orb version

2.4.0

Current behavior

But issue looks to be present on newer versions too.

Bitbucket changed ssh host keys yesterday.

The bitbucket keys listed here and here seem to be incorrect now. Causing pipelines to fail with a message

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed

this guide might be helpful:
https://bitbucket.org/blog/ssh-host-key-changes

Minimum reproduction config

N/A

Other

No response

BitBucket & GitLab test

There are GitHub test but no bitbucket test. This is due to I don't have account and needs to do.

But it may help when there's fingerprint issue.

stopped CI pipeline when macOS test is enabled

I don't have any idea why...

https://app.circleci.com/pipelines/github/guitarrapc/git-shallow-clone-orb/217/workflows/0c42b386-5274-479f-a276-9f0e2eb9e44b

Please add an `allow_fail` option

Is there an existing issue that is already proposing this?

I have searched the existing issues

Describe the problem imposed by not having this feature

Sometimes we merge a PR without waiting for its CircleCI workflow to complete. When we merge, we delete the source branch. Then the subsequent job in that workflow fails on the git-shallow-clone/checkout step because the branch doesn't exist...

We would like to have a solution to prevent such workflows from spoiling our CI statistics.

Describe the solution you'd like

I propose to add an allow_fail option, set to false by default. If set to true, then the whole step would exit with code 0 even if anything would go wrong in it.

We would set it for our non-critical jobs, such as sending test coverage info to our coverage server.

Shallow cloning fails when running on pull request

When I run the API for a pull request:

curl -X POST --location "https://circleci.com/api/v2/project/gh/storybookjs/storybook/pipeline" \
              -H "Content-Type: application/json" \
              -H "Circle-Token: $CIRCLE_CI_TOKEN" \
              -d '{
                    "branch": "pull/19768/head",
                    "parameters": {
                      "workflow": "pr"
                    }
                  }'

I get an error in the shallow clone:

+ '[' /home/circleci = / ']'
+ mkdir -p /home/circleci/.ssh
++ command -v ssh-keyscan
+ '[' -x /usr/bin/ssh-keyscan ']'
+ '[' false == true ']'
+ '[' false == true ']'
+ '[' false '!=' true ']'
+ echo 'github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==
'
+ '[' false '!=' true ']'
+ echo 'bitbucket.org ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAubiN81eDcafrgMeLzaFPsw2kNvEcqTKl/VqLat/MaB33pZy0y3rJZtnqwR2qOOvbwKZYKiEO1O6VqNEBxKvJJelCq0dTXWT5pbO2gDXC6h6QDXCaHo6pOHGPUy+YBaGQRGuSusMEASYiWunYN0vCAI8QaXnWMXNMdFP3jHAJH0eDsoiGnLPBlBp4TNm6rYI74nMzgz3B9IikW4WVK+dc8KZJZWYjAuORU3jc1c/NPskD2ASinf8v3xnfXeukU0sJ5N6m5E8VLjObPEO+mN2t/FZTMZLiFqPWc/ALSqnMnnhwrNi2rbfg/rd/IpL8Le3pSBne8+seeFVBoGqzHM9yXw==
'
+ umask 077
+ touch /home/circleci/.ssh/id_rsa
+ chmod 0600 /home/circleci/.ssh/id_rsa
+ echo
+ git config --global url.ssh://[email protected] https://github.com
+ git config --global gc.auto 0
+ git clone --depth 1 --verbose [email protected]:storybookjs/storybook.git .
Cloning into '.'...
Warning: Permanently added the RSA host key for IP address '140.82.113.4' to the list of known hosts.

remote: Enumerating objects: 4644, done.        
remote: Counting objects:   0% (1/4644)        
remote: Counting objects:   1% (47/4644)        
remote: Counting objects:   2% (93/4644)        
remote: Counting objects:   3% (140/4644)        
remote: Counting objects:   4% (186/4644)        
remote: Counting objects:   5% (233/4644)        
remote: Counting objects:   6% (279/4644)        
remote: Counting objects:   7% (326/4644)        
remote: Counting objects:   8% (372/4644)        
remote: Counting objects:   9% (418/4644)        
remote: Counting objects:  10% (465/4644)        
remote: Counting objects:  11% (511/4644)        
remote: Counting objects:  12% (558/4644)        
remote: Counting objects:  13% (604/4644)        
remote: Counting objects:  14% (651/4644)        
remote: Counting objects:  15% (697/4644)        
remote: Counting objects:  16% (744/4644)        
remote: Counting objects:  17% (790/4644)        
remote: Counting objects:  18% (836/4644)        
remote: Counting objects:  19% (883/4644)        
remote: Counting objects:  20% (929/4644)        
remote: Counting objects:  21% (976/4644)        
remote: Counting objects:  22% (1022/4644)        
remote: Counting objects:  23% (1069/4644)        
remote: Counting objects:  24% (1115/4644)        
remote: Counting objects:  25% (1161/4644)        
remote: Counting objects:  26% (1208/4644)        
remote: Counting objects:  27% (1254/4644)        
remote: Counting objects:  28% (1301/4644)        
remote: Counting objects:  29% (1347/4644)        
remote: Counting objects:  30% (1394/4644)        
remote: Counting objects:  31% (1440/4644)        
remote: Counting objects:  32% (1487/4644)        
remote: Counting objects:  33% (1533/4644)        
remote: Counting objects:  34% (1579/4644)        
remote: Counting objects:  35% (1626/4644)        
remote: Counting objects:  36% (1672/4644)        
remote: Counting objects:  37% (1719/4644)        
remote: Counting objects:  38% (1765/4644)        
remote: Counting objects:  39% (1812/4644)        
remote: Counting objects:  40% (1858/4644)        
remote: Counting objects:  41% (1905/4644)        
remote: Counting objects:  42% (1951/4644)        
remote: Counting objects:  43% (1997/4644)        
remote: Counting objects:  44% (2044/4644)        
remote: Counting objects:  45% (2090/4644)        
remote: Counting objects:  46% (2137/4644)        
remote: Counting objects:  47% (2183/4644)        
remote: Counting objects:  48% (2230/4644)        
remote: Counting objects:  49% (2276/4644)        
remote: Counting objects:  50% (2322/4644)        
remote: Counting objects:  51% (2369/4644)        
remote: Counting objects:  52% (2415/4644)        
remote: Counting objects:  53% (2462/4644)        
remote: Counting objects:  54% (2508/4644)        
remote: Counting objects:  55% (2555/4644)        
remote: Counting objects:  56% (2601/4644)        
remote: Counting objects:  57% (2648/4644)        
remote: Counting objects:  58% (2694/4644)        
remote: Counting objects:  59% (2740/4644)        
remote: Counting objects:  60% (2787/4644)        
remote: Counting objects:  61% (2833/4644)        
remote: Counting objects:  62% (2880/4644)        
remote: Counting objects:  63% (2926/4644)        
remote: Counting objects:  64% (2973/4644)        
remote: Counting objects:  65% (3019/4644)        
remote: Counting objects:  66% (3066/4644)        
remote: Counting objects:  67% (3112/4644)        
remote: Counting objects:  68% (3158/4644)        
remote: Counting objects:  69% (3205/4644)        
remote: Counting objects:  70% (3251/4644)        
remote: Counting objects:  71% (3298/4644)        
remote: Counting objects:  72% (3344/4644)        
remote: Counting objects:  73% (3391/4644)        
remote: Counting objects:  74% (3437/4644)        
remote: Counting objects:  75% (3483/4644)        
remote: Counting objects:  76% (3530/4644)        
remote: Counting objects:  77% (3576/4644)        
remote: Counting objects:  78% (3623/4644)        
remote: Counting objects:  79% (3669/4644)        
remote: Counting objects:  80% (3716/4644)        
remote: Counting objects:  81% (3762/4644)        
remote: Counting objects:  82% (3809/4644)        
remote: Counting objects:  83% (3855/4644)        
remote: Counting objects:  84% (3901/4644)        
remote: Counting objects:  85% (3948/4644)        
remote: Counting objects:  86% (3994/4644)        
remote: Counting objects:  87% (4041/4644)        
remote: Counting objects:  88% (4087/4644)        
remote: Counting objects:  89% (4134/4644)        
remote: Counting objects:  90% (4180/4644)        
remote: Counting objects:  91% (4227/4644)        
remote: Counting objects:  92% (4273/4644)        
remote: Counting objects:  93% (4319/4644)        
remote: Counting objects:  94% (4366/4644)        
remote: Counting objects:  95% (4412/4644)        
remote: Counting objects:  96% (4459/4644)        
remote: Counting objects:  97% (4505/4644)        
remote: Counting objects:  98% (4552/4644)        
remote: Counting objects:  99% (4598/4644)        
remote: Counting objects: 100% (4644/4644)        
remote: Counting objects: 100% (4644/4644), done.        
remote: Compressing objects:   0% (1/3533)        
remote: Compressing objects:   1% (36/3533)        
remote: Compressing objects:   2% (71/3533)        
remote: Compressing objects:   3% (106/3533)        
remote: Compressing objects:   4% (142/3533)        
remote: Compressing objects:   5% (177/3533)        
remote: Compressing objects:   6% (212/3533)        
remote: Compressing objects:   7% (248/3533)        
remote: Compressing objects:   8% (283/3533)        
remote: Compressing objects:   9% (318/3533)        
remote: Compressing objects:  10% (354/3533)        
remote: Compressing objects:  11% (389/3533)        
remote: Compressing objects:  12% (424/3533)        
remote: Compressing objects:  13% (460/3533)        
remote: Compressing objects:  14% (495/3533)        
remote: Compressing objects:  15% (530/3533)        
remote: Compressing objects:  16% (566/3533)        
remote: Compressing objects:  17% (601/3533)        
remote: Compressing objects:  18% (636/3533)        
remote: Compressing objects:  19% (672/3533)        
remote: Compressing objects:  20% (707/3533)        
remote: Compressing objects:  21% (742/3533)        
remote: Compressing objects:  22% (778/3533)        
remote: Compressing objects:  23% (813/3533)        
remote: Compressing objects:  24% (848/3533)        
remote: Compressing objects:  25% (884/3533)        
remote: Compressing objects:  26% (919/3533)        
remote: Compressing objects:  27% (954/3533)        
remote: Compressing objects:  28% (990/3533)        
remote: Compressing objects:  29% (1025/3533)        
remote: Compressing objects:  30% (1060/3533)        
remote: Compressing objects:  31% (1096/3533)        
remote: Compressing objects:  32% (1131/3533)        
remote: Compressing objects:  33% (1166/3533)        
remote: Compressing objects:  34% (1202/3533)        
remote: Compressing objects:  35% (1237/3533)        
remote: Compressing objects:  36% (1272/3533)        
remote: Compressing objects:  37% (1308/3533)        
remote: Compressing objects:  38% (1343/3533)        
remote: Compressing objects:  39% (1378/3533)        
remote: Compressing objects:  40% (1414/3533)        
remote: Compressing objects:  41% (1449/3533)        
remote: Compressing objects:  42% (1484/3533)        
remote: Compressing objects:  43% (1520/3533)        
remote: Compressing objects:  44% (1555/3533)        
remote: Compressing objects:  45% (1590/3533)        
remote: Compressing objects:  46% (1626/3533)        
remote: Compressing objects:  47% (1661/3533)        
remote: Compressing objects:  48% (1696/3533)        
remote: Compressing objects:  49% (1732/3533)        
remote: Compressing objects:  50% (1767/3533)        
remote: Compressing objects:  51% (1802/3533)        
remote: Compressing objects:  52% (1838/3533)        
remote: Compressing objects:  53% (1873/3533)        
remote: Compressing objects:  54% (1908/3533)        
remote: Compressing objects:  55% (1944/3533)        
remote: Compressing objects:  56% (1979/3533)        
remote: Compressing objects:  57% (2014/3533)        
remote: Compressing objects:  58% (2050/3533)        
remote: Compressing objects:  59% (2085/3533)        
remote: Compressing objects:  60% (2120/3533)        
remote: Compressing objects:  61% (2156/3533)        
remote: Compressing objects:  62% (2191/3533)        
remote: Compressing objects:  63% (2226/3533)        
remote: Compressing objects:  64% (2262/3533)        
remote: Compressing objects:  65% (2297/3533)        
remote: Compressing objects:  66% (2332/3533)        
remote: Compressing objects:  67% (2368/3533)        
remote: Compressing objects:  68% (2403/3533)        
remote: Compressing objects:  69% (2438/3533)        
remote: Compressing objects:  70% (2474/3533)        
remote: Compressing objects:  71% (2509/3533)        
remote: Compressing objects:  72% (2544/3533)        
remote: Compressing objects:  73% (2580/3533)        
remote: Compressing objects:  74% (2615/3533)        
remote: Compressing objects:  75% (2650/3533)        
remote: Compressing objects:  76% (2686/3533)        
remote: Compressing objects:  77% (2721/3533)        
remote: Compressing objects:  78% (2756/3533)        
remote: Compressing objects:  79% (2792/3533)        
remote: Compressing objects:  80% (2827/3533)        
remote: Compressing objects:  81% (2862/3533)        
remote: Compressing objects:  82% (2898/3533)        
remote: Compressing objects:  83% (2933/3533)        
remote: Compressing objects:  84% (2968/3533)        
remote: Compressing objects:  85% (3004/3533)        
remote: Compressing objects:  86% (3039/3533)        
remote: Compressing objects:  87% (3074/3533)        
remote: Compressing objects:  88% (3110/3533)        
remote: Compressing objects:  89% (3145/3533)        
remote: Compressing objects:  90% (3180/3533)        
remote: Compressing objects:  91% (3216/3533)        
remote: Compressing objects:  92% (3251/3533)        
remote: Compressing objects:  93% (3286/3533)        
remote: Compressing objects:  94% (3322/3533)        
remote: Compressing objects:  95% (3357/3533)        
remote: Compressing objects:  96% (3392/3533)        
remote: Compressing objects:  97% (3428/3533)        
remote: Compressing objects:  98% (3463/3533)        
remote: Compressing objects:  99% (3498/3533)        
remote: Compressing objects: 100% (3533/3533)        
remote: Compressing objects: 100% (3533/3533), done.        
Receiving objects:   0% (1/4644)
Receiving objects:   1% (47/4644)
Receiving objects:   2% (93/4644)
Receiving objects:   3% (140/4644)
Receiving objects:   4% (186/4644)
Receiving objects:   5% (233/4644)
Receiving objects:   6% (279/4644)
Receiving objects:   7% (326/4644)
Receiving objects:   8% (372/4644)
Receiving objects:   9% (418/4644)
Receiving objects:  10% (465/4644)
Receiving objects:  11% (511/4644)
Receiving objects:  12% (558/4644)
Receiving objects:  13% (604/4644)
Receiving objects:  14% (651/4644)
Receiving objects:  15% (697/4644)
Receiving objects:  16% (744/4644)
Receiving objects:  17% (790/4644)
Receiving objects:  18% (836/4644)
Receiving objects:  19% (883/4644)
Receiving objects:  20% (929/4644)
Receiving objects:  21% (976/4644)
Receiving objects:  22% (1022/4644)
Receiving objects:  23% (1069/4644)
Receiving objects:  24% (1115/4644)
Receiving objects:  25% (1161/4644)
Receiving objects:  26% (1208/4644)
Receiving objects:  27% (1254/4644)
Receiving objects:  28% (1301/4644)
Receiving objects:  29% (1347/4644)
Receiving objects:  30% (1394/4644)
Receiving objects:  31% (1440/4644)
Receiving objects:  32% (1487/4644)
Receiving objects:  33% (1533/4644)
Receiving objects:  34% (1579/4644)
Receiving objects:  35% (1626/4644)
Receiving objects:  36% (1672/4644)
Receiving objects:  37% (1719/4644)
Receiving objects:  38% (1765/4644)
Receiving objects:  39% (1812/4644)
Receiving objects:  40% (1858/4644)
Receiving objects:  41% (1905/4644)
Receiving objects:  42% (1951/4644)
Receiving objects:  43% (1997/4644)
Receiving objects:  44% (2044/4644)
Receiving objects:  45% (2090/4644)
Receiving objects:  46% (2137/4644)
Receiving objects:  47% (2183/4644)
Receiving objects:  48% (2230/4644)
Receiving objects:  49% (2276/4644)
Receiving objects:  50% (2322/4644)
Receiving objects:  51% (2369/4644)
Receiving objects:  52% (2415/4644)
Receiving objects:  53% (2462/4644)
Receiving objects:  54% (2508/4644)
Receiving objects:  55% (2555/4644)
Receiving objects:  56% (2601/4644)
Receiving objects:  57% (2648/4644)
Receiving objects:  58% (2694/4644)
Receiving objects:  59% (2740/4644)
Receiving objects:  60% (2787/4644)
Receiving objects:  61% (2833/4644)
Receiving objects:  62% (2880/4644)
Receiving objects:  63% (2926/4644)
Receiving objects:  64% (2973/4644)
Receiving objects:  65% (3019/4644)
Receiving objects:  66% (3066/4644)
Receiving objects:  67% (3112/4644)
Receiving objects:  68% (3158/4644)
Receiving objects:  69% (3205/4644)
Receiving objects:  70% (3251/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  71% (3298/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  72% (3344/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  73% (3391/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  74% (3437/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  75% (3483/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  76% (3530/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  77% (3576/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  78% (3623/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  79% (3669/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  80% (3716/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  81% (3762/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  82% (3809/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  83% (3855/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  84% (3901/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  85% (3948/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  86% (3994/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  87% (4041/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  88% (4087/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  89% (4134/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  90% (4180/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  91% (4227/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  92% (4273/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  93% (4319/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  94% (4366/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  95% (4412/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  96% (4459/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  97% (4505/4644), 17.12 MiB | 34.23 MiB/s
remote: Total 4644 (delta 1141), reused 2203 (delta 859), pack-reused 0        
Receiving objects:  98% (4552/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects:  99% (4598/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects: 100% (4644/4644), 17.12 MiB | 34.23 MiB/s
Receiving objects: 100% (4644/4644), 34.48 MiB | 38.76 MiB/s, done.
Resolving deltas:   0% (0/1141)
Resolving deltas:   1% (12/1141)
Resolving deltas:   2% (23/1141)
Resolving deltas:   3% (36/1141)
Resolving deltas:   4% (46/1141)
Resolving deltas:   5% (62/1141)
Resolving deltas:   6% (69/1141)
Resolving deltas:   7% (80/1141)
Resolving deltas:   8% (93/1141)
Resolving deltas:   9% (103/1141)
Resolving deltas:  10% (115/1141)
Resolving deltas:  11% (126/1141)
Resolving deltas:  12% (137/1141)
Resolving deltas:  13% (151/1141)
Resolving deltas:  14% (162/1141)
Resolving deltas:  15% (172/1141)
Resolving deltas:  16% (183/1141)
Resolving deltas:  17% (194/1141)
Resolving deltas:  18% (206/1141)
Resolving deltas:  19% (219/1141)
Resolving deltas:  20% (229/1141)
Resolving deltas:  21% (240/1141)
Resolving deltas:  22% (252/1141)
Resolving deltas:  23% (264/1141)
Resolving deltas:  24% (274/1141)
Resolving deltas:  25% (286/1141)
Resolving deltas:  26% (298/1141)
Resolving deltas:  27% (309/1141)
Resolving deltas:  28% (320/1141)
Resolving deltas:  29% (331/1141)
Resolving deltas:  30% (343/1141)
Resolving deltas:  31% (354/1141)
Resolving deltas:  32% (366/1141)
Resolving deltas:  33% (377/1141)
Resolving deltas:  34% (388/1141)
Resolving deltas:  35% (400/1141)
Resolving deltas:  36% (412/1141)
Resolving deltas:  37% (423/1141)
Resolving deltas:  38% (434/1141)
Resolving deltas:  39% (447/1141)
Resolving deltas:  40% (457/1141)
Resolving deltas:  41% (470/1141)
Resolving deltas:  42% (480/1141)
Resolving deltas:  43% (491/1141)
Resolving deltas:  44% (503/1141)
Resolving deltas:  45% (514/1141)
Resolving deltas:  46% (526/1141)
Resolving deltas:  47% (537/1141)
Resolving deltas:  48% (548/1141)
Resolving deltas:  49% (560/1141)
Resolving deltas:  50% (572/1141)
Resolving deltas:  51% (582/1141)
Resolving deltas:  52% (594/1141)
Resolving deltas:  53% (605/1141)
Resolving deltas:  54% (617/1141)
Resolving deltas:  55% (628/1141)
Resolving deltas:  56% (639/1141)
Resolving deltas:  57% (651/1141)
Resolving deltas:  58% (663/1141)
Resolving deltas:  59% (674/1141)
Resolving deltas:  60% (685/1141)
Resolving deltas:  61% (698/1141)
Resolving deltas:  62% (708/1141)
Resolving deltas:  63% (719/1141)
Resolving deltas:  64% (731/1141)
Resolving deltas:  65% (743/1141)
Resolving deltas:  66% (754/1141)
Resolving deltas:  67% (765/1141)
Resolving deltas:  68% (776/1141)
Resolving deltas:  69% (788/1141)
Resolving deltas:  70% (801/1141)
Resolving deltas:  71% (811/1141)
Resolving deltas:  72% (823/1141)
Resolving deltas:  73% (834/1141)
Resolving deltas:  74% (845/1141)
Resolving deltas:  75% (856/1141)
Resolving deltas:  76% (868/1141)
Resolving deltas:  77% (881/1141)
Resolving deltas:  78% (890/1141)
Resolving deltas:  79% (902/1141)
Resolving deltas:  80% (913/1141)
Resolving deltas:  81% (925/1141)
Resolving deltas:  82% (938/1141)
Resolving deltas:  83% (948/1141)
Resolving deltas:  84% (959/1141)
Resolving deltas:  85% (971/1141)
Resolving deltas:  86% (982/1141)
Resolving deltas:  87% (993/1141)
Resolving deltas:  88% (1005/1141)
Resolving deltas:  89% (1016/1141)
Resolving deltas:  90% (1027/1141)
Resolving deltas:  91% (1039/1141)
Resolving deltas:  92% (1050/1141)
Resolving deltas:  93% (1062/1141)
Resolving deltas:  94% (1074/1141)
Resolving deltas:  95% (1085/1141)
Resolving deltas:  96% (1096/1141)
Resolving deltas:  97% (1107/1141)
Resolving deltas:  98% (1119/1141)
Resolving deltas:  99% (1130/1141)
Resolving deltas: 100% (1141/1141)
Resolving deltas: 100% (1141/1141), done.
+ cd .
+ '[' -n '' ']'
++ echo pull/19768/head
++ grep -e '^pull/*'
+ [[ -n pull/19768/head ]]
+ git fetch --depth 10 --force origin pull/19768/head/head:remotes/origin/pull/19768/head
fatal: couldn't find remote ref pull/19768/head/head

Exited with code exit status 128

This line seems to be the problem:
https://github.com/guitarrapc/git-shallow-clone-orb/blob/master/src/commands/checkout_advanced.yml#L103

I also tried calling the API with: "branch": "pull/19768" instead of "branch": "pull/19768" but that gives me a branch not found error. (400)

[Feature Request] Add an option "path" same as the built in step "checkout"

checkout

Thank you for your great project.
This orb is very useful!

I have a feature request to add an option "path" same as the built in step "checkout".
It is useful we can change the destination where the repository is cloned.

Thank you.

Bug: Bitbucket RSA host key rotation

Is there an existing issue for this?

I have searched the existing issues

Orb version

2.5.0

Current behavior

Checkout of bitbucket repo fails. Bitbucket is rotating the host keys (https://bitbucket.org/blog/ssh-host-key-changes)
The keys in the orb need to be updated with the new ones provided from bitbucket

Minimum reproduction config

any bitbucket checkout

Other

No response

Reasoning behind separate clone and checkout?

Is there an existing issue that is already proposing this?

I have searched the existing issues

Describe the problem imposed by not having this feature

N/A

Describe the solution you'd like

N/A

Other

Hi, this is not really a feature request, it's just a question. (Probably more suited to a GitHub Discussion.) The question is: is there a reasoning behind implementing shallow clone/checkout with two different commands? E.g.:

git clone ${clone_tag_args} --depth << parameters.depth >>

if [ -n "$CIRCLE_TAG" ]; then
          # tag
          git fetch ${fetch_tag_args} --depth << parameters.fetch_depth >> --force origin "+refs/tags/${CIRCLE_TAG}:refs/tags/${CIRCLE_TAG}"
        elif [[ $(echo $CIRCLE_BRANCH | grep -E ^pull\/[0-9]+$) ]]; then
          # pull request
          git fetch ${fetch_tag_args} --depth << parameters.fetch_depth >> --force origin "${CIRCLE_BRANCH}/head:remotes/origin/${CIRCLE_BRANCH}"
        else
          # others
          git fetch ${fetch_tag_args} --depth=<< parameters.fetch_depth >> --force origin "$CIRCLE_BRANCH:remotes/origin/$CIRCLE_BRANCH"
        fi

I'm asking because–wouldn't it be simpler to do it with only a single command? I.e.:

git clone --depth $depth --branch $branch_or_tag $repo_url

Is there a corner case I'm missing, that is handled by doing clone and checkout separately?

github RSA host key is updated.

hi @guitarrapc

https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/

github.com RSA host key is replaced today.
should update the echo line of code to new RSA key.

git-shallow-clone-orb/src/commands/checkout.yml

Line 71 in 39f6ba8

  echo 'github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ== 

Request: Support Source caching

Is there an existing issue that is already proposing this?

I have searched the existing issues

Describe the problem imposed by not having this feature

When using shallow checkout in addition with the source caching in CCI https://circleci.com/docs/caching/#source-caching the clone fails because the folder isn't empty. This is because the .git folder is restored from a previous build.

The benefits of this is that the shallow checkout generates cache much smaller than the regular checkout (in our case 347MB vs 940MB) and it speeds up a lot the checkout process and cache recovering, consuming less credits as well

Describe the solution you'd like

If you analyze the source code of the default CCI checkout step you can see that they have a condition for this, and instead of cloning the repo, they do a "remote set-url"

if [ -e '/Users/distiller/project/.git' ] ; then
echo 'Fetching into existing repository'
existing_repo='true'
cd '/Users/distiller/project'
git remote set-url origin "$CIRCLE_REPOSITORY_URL" || true
else
echo 'Cloning git repository'
existing_repo='false'
mkdir -p '/Users/distiller/project'
cd '/Users/distiller/project'
git clone --no-checkout "$CIRCLE_REPOSITORY_URL" .
fi

Could you please add support to your orb to the source-caching provided by CCI?

Thank you

Other

No response

git-lfs support?

Would be nice to have some flag to also fetch git lfs content.

Schedule test

Current test will run when PUSH or PR happen. Schedule test let me know if there any issue event code have no change.

Error when using `path` option in parent of working_directory

First I tried to replace the "normal" checkout step with the following

working_directory: ~/project/api

steps:
  - unless:
          condition:
            or:
              - equal: [ true, << pipeline.parameters.ci-changed >> ]
              - equal: [ true, << pipeline.parameters.api-changed >> ]
              - equal: [ true, << pipeline.parameters.common-changed >> ]
          steps:
            - run: 
                name: skip job b/c no changes occured
                command: circleci-agent step halt
  - checkout:
      path: ~/project
  - restore_cache:
      keys:
        - v1-dependencies-api-{{ checksum "yarn.lock" }}
        - v1-dependencies-api-

working_directory: ~/project/api

steps:
  - unless:
          condition:
            or:
              - equal: [ true, << pipeline.parameters.ci-changed >> ]
              - equal: [ true, << pipeline.parameters.api-changed >> ]
              - equal: [ true, << pipeline.parameters.common-changed >> ]
          steps:
            - run: 
                name: skip job b/c no changes occured
                command: circleci-agent step halt
  - git-shallow-clone/checkout:
      path: ~/project
  - restore_cache:
      keys:
        - v1-dependencies-api-{{ checksum "yarn.lock" }}
        - v1-dependencies-api-

But it seems the orb cannot handle the ~/project path correctly and checks out the repo into the wrong folder.
My next step (restore_cache) failed with:

error computing cache key: template: cacheKey:1:23: executing "cacheKey" at <checksum "yarn.lock">: error calling checksum: open /home/circleci/project/api/yarn.lock: no such file or directory

So I changed the path to an absolute path like this:

working_directory: ~/project/api

steps:
  - unless:
          condition:
            or:
              - equal: [ true, << pipeline.parameters.ci-changed >> ]
              - equal: [ true, << pipeline.parameters.api-changed >> ]
              - equal: [ true, << pipeline.parameters.common-changed >> ]
          steps:
            - run: 
                name: skip job b/c no changes occured
                command: circleci-agent step halt
  - git-shallow-clone/checkout:
      path: /home/circleci/project

Now I get the error message: fatal: destination path '/home/circleci/project' already exists and is not an empty directory.

Logging into the machine via ssh and checking the folder show that the ~/project/api was already created (by CircleCI!?).
How does the "normal" checkout command handle this case?

Can we somehow improve the git-shallow-clone/checkout command to handle:

path with ~/... in it
the "path already exists" error

Maybe you have a quick idea or I can try and take a look at how to tackle this.

Bug: shallow checkout if the branch has moved too much foward

Is there an existing issue for this?

I have searched the existing issues

Orb version

2.5.0

Current behavior

I use the simple checkout command in my job:

steps:
  - git-shallow-clone/checkout

It is part of a workflow that springs on my main branch and is gated by an approval in Circle CI which allows me to start this workflow only on some commits of my branch. However if I try to approve it for an older commit, then the shallow clone fails.

I believe it should not try to start the fetch from the current branch, but from the Circle SHA1, or maybe at least add an option where one can checkout only the code at CIRCLE_SHA1 version without caring about the branch anymore.

Minimum reproduction config

---
version: 2.1
orbs:
  git-shallow-clone: guitarrapc/[email protected]
workflows:
  release:
    jobs:
      - manual-approval:
          only-on-branch: main
          type: approval
          filters:
            branches:
              only:
                - main
      - start-release:
          requires: manual-approval
  jobs:
    start-release:
      executor: python/default
      steps:
        - git-shallow-clone/checkout

Add this config above, then create 20+ additional commits on the main branch, then go back in CircleCI to approve the "manual-approval" job of the release workflow for an old commit of the main branch.

Other

No response

Future of git-shallow-clone-orb

Official annoucement had been added to CircleCI Ideas "
Allow for Shallow Clone command in 2.0" which state planning offer an blobless clone. What a great news to all CircleCI users include this orb!

I'm very welcome about new checkout coming, and it means this repository may deprecate when it comes to GA and stable. I will keep maintain this orb and add other checkout options, like sparse checkout, but let's wait for next news.

Thank you to all.
Best,

enable macOS build

Apologies! Your build didn’t run because you haven’t selected a plan that includes macOS yet. Please change to a Performance Plan. If you have any questions please feel free to contact support.

Oh my...

[Feature Request] Support the parameter "shallow_since"

git clone supports the option --shallow-since.

--shallow-since=<date>
Create a shallow clone with a history after the specified time.

To support the option, I propose to add the optional parameter shallow_since.
By default this parameter is empty and --shallow-since isn't set, so this change keeps the compatibility.
If this parameter is set, the parameter --shallow-since is set and the parameter depth is ignored.

What do you think?

Thank you.

Why not respecting `working_directory` configuration since v2.7.0?

If working_directory parameter is configured in CircleCI config.yml

executors:
  node:
    docker:
      - image: cimg/node:18.17.1
    working_directory: ~/my-original-working-directory

and use of git-shallow-clone checkout without path parameter.

  frontend_test:
    executor:
      name: node
    steps:
      - git-shallow-clone/checkout
      ...

thus, git-shallow-clone/checkout will checkout source codes to ~/project
but our steps will do something in ~/my-original-working-directory that is empty directory.

circleci@965ef695fc70:~$ ls
bin  my-original-working-directory  project
circleci@965ef695fc70:~$ ls my-original-working-directory
circleci@965ef695fc70:~$ ls project/
<our_source_code_files...>

Why are you using hard coded path of ~/project since v2.7.0?

I'm thinking support of both respecting working_directory configuration and SourceCaching is possible by using $CIRCLE_WORKING_DIRECTORY environment variable instead of hard-coded ${HOME}/project.

https://circleci.com/docs/configuration-reference/#job-name:~:text=Processes%20run%20during%20the%20job%20can%20use%20the%20%24CIRCLE_WORKING_DIRECTORY%20environment%20variable%20to%20refer%20to%20this%20directory.

What do you think?

Cannot find a definition for job named git/checkout_shallow

I tried the example but unfortunately keep getting an error:

version: 2.1

orbs:
  git: guitarrapc/[email protected]

workflows:
  build:
    jobs:
      - git/checkout_shallow

Error calling workflow: build
Cannot find a definition for job named git/checkout_shallow

feat: Support to set `git fetch` command options

Similar to #9 .

git fetch command supports some options about shallow clone such as --shallow-since.

       --deepen=<depth>
           Similar to --depth, except it specifies the number of commits from the current
           shallow boundary instead of from the tip of each remote branch history.

       --shallow-since=<date>
           Deepen or shorten the history of a shallow repository to include all reachable
           commits after <date>.

       --shallow-exclude=<revision>
           Deepen or shorten the history of a shallow repository to exclude commits
           reachable from a specified remote branch or tag. This option can be specified
           multiple times.

So I propose to add an optional option fetch_options.
When the option fetch_options is set the parameter fetch_depth is ignored because --depth argument may conflict other arguments.
If users want to set the depth parameter along with fetch_options,
they can set --depth argument in fetch_options.
--force option shouldn't be set in fetch_options because it is set automatically.
And in case of a pull request, -t option is also set automatically.

What do you think?

Thank you.

Bug:

Is there an existing issue for this?

I have searched the existing issues

Orb version

2.4.1

Current behavior

I started getting such message

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@       WARNING: POSSIBLE DNS SPOOFING DETECTED!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
The RSA host key for github.com has changed,
and the key for the corresponding IP address
is unknown. This could either mean that
DNS SPOOFING is happening or the IP address for the host
and its host key have changed at the same time.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.

Switching to regular circle ci checkout it's gone

Minimum reproduction config

orbs:
    docker: circleci/[email protected]
    slack: circleci/[email protected]
    git-shallow-clone: guitarrapc/[email protected]
jobs:
    lint:
        working_directory: ~/repo
        steps:
            - git-shallow-clone/checkout:
                  depth: 1

guitarrapc / git-shallow-clone-orb Goto Github PK

git-shallow-clone-orb's Introduction

Git-shallow-clone-orb

Usage

Contributing

Publish

Add Test

Basic orb setup

git-shallow-clone-orb's People

Contributors

Stargazers

Watchers

Forkers

git-shallow-clone-orb's Issues

How often

Failed log

Is there an existing issue for this?

Orb version

Current behavior

Minimum reproduction config

Other

Is there an existing issue that is already proposing this?

Describe the problem imposed by not having this feature

Describe the solution you'd like

Is there an existing issue for this?

Orb version

Current behavior

Minimum reproduction config

Other

Is there an existing issue that is already proposing this?

Describe the problem imposed by not having this feature

Describe the solution you'd like

Other

Is there an existing issue that is already proposing this?

Describe the problem imposed by not having this feature

Describe the solution you'd like

Other

Is there an existing issue for this?

Orb version

Current behavior

Minimum reproduction config

Other

Is there an existing issue for this?

Orb version

Current behavior

Minimum reproduction config

Other

Recommend Projects

Recommend Topics

Recommend Org