Giter Club home page Giter Club logo

source-indexer's Introduction

source-indexer

This repo contains the code for building http://source.dot.net

Build Status

Build Status

What Is It?

This repo uses https://github.com/KirillOsenkov/SourceBrowser (with a few additions here https://github.com/dotnet/SourceBrowser/tree/source-indexer) to index the dotnet sources and produce a navigatable and searchable website containing the full source code. This includes code from the runtime, winforms, wpf, aspnetcore, and msbuild, among others. For a full list see here https://github.com/dotnet/source-indexer/blob/main/src/index/repositories.props.

Build Prerequsites

The build requires .NET 8.0 and Visual Studio 2022 to build.

Build

The build will only work on windows because the source indexer executable is a .net framework executable.

  1. git clone https://github.com/dotnet/source-indexer.git
  2. For each *.sln file dotnet restore
  3. Find VS 2022 msbuild.exe on your machine, typically found at C:\Program Files (x86)\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\MSBuild.exe
  4. msbuild build.proj

Running the built index

After the build is finished the index will exist in bin\index and can be run by running dotnet Microsoft.SourceBrowser.SourceIndexServer.dll in that folder. The index will be served on http://localhost:5000

Deployment

The index is deployed by the VSTS build to the netsourceindex azure app service, with the index data stored in the netsourceindex storage account. The deployment does the following things.

  1. Split the generated index from the binaries and static data for the website.
  2. Upload the generated index into a new container in the netsourceindex storage account.
  3. Deploy the binaries and static data to the staging slot of the app service.
  4. Update the app service settings with the url of the storage container the index data was uploaded to
  5. Restart the app service
  6. Test the application by performing a GET of the url, fail if it doesn't return 200 OK
  7. Swap the staging slot into production for the app service
  8. Delete storage containers that haven't been used by the app service in the last 10 builds.

Monitoring

https://source.dot.net is monitored using availability tests from the dotnet-eng application insights resource. Alerting is handled through grafana here https://dotnet-eng-grafana.westus2.cloudapp.azure.com/d/arcadeAvailability/service-availability?orgId=1&refresh=30s

source-indexer's People

Contributors

akoeplinger avatar alexperovich avatar andriysvyryd avatar carlossanlop avatar dagood avatar danmoseley avatar davidfowl avatar dazombiekiller avatar dependabot[bot] avatar directhex avatar eerhardt avatar ericstj avatar garath avatar hoyosjs avatar i3arnon avatar johntortugo avatar jonfortescue avatar joperezr avatar kirillosenkov avatar mattgal avatar radical avatar sebastienros avatar stephentoub avatar tannergooding avatar terrajobst avatar viktorhofer avatar wtgodbe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

source-indexer's Issues

ICollectionViewLiveShaping is missing

Note: This issue also affects two other interfaces implemented by ListCollectionView: IEditableCollectionViewAddNewItem and IItemProperties

Consider the type ListCollectionView (.NET Source Browser) (GitHub) (GitHub link was retrieved using the "Web Access" link at the bottom of the page in the .NET Source Browser)

ListCollectionView implements ICollectionViewLiveShaping

  • Clicking on the interface name in the .NET Source Browser navigates to a blank page
  • Searching for the interface name in the .NET Source Browser shows "No results found"
  • Clicking on the interface name in GitHub navigates to ICollectionViewLiveShaping.cs

Consider displaying PAL codes for all platforms

#47 (comment)

source.dot.net is only able to show one platform's code. By design that's currently Linux. Hopefully in the future that restriction will be lifted, likely via multiple hosts.

It'd be lovely if this could be done; Not having to manually navigate between codes through GitHub / clone and build locally would be very nice.

Completely broken.

Trying to access basic types such as "String" or "Stream" fails. All it finds is a load of tests. Almost nothing is accessible.

AspNetCore source code no longer found

Since a day or two, the source code of AspNetCore is no longer found on source.dot.net web site. Even providing the exact url returns no result. For example, trying to search ComponentBase (for Blazor framework) returns nothing.

Winforms indexing is of its ref not its src

Search for anything in winforms eg https://source.dot.net/#q=datagridv and you always get the ref file.

I cannot figure out why since the repositories.props says src/*/src/*.csproj; and winforms is laid out like

c:\git\winforms\src\System.Windows.Forms\ref\System.Windows.Forms.Ref.csproj
c:\git\winforms\src\System.Windows.Forms\src\System.Windows.Forms.csproj
c:\git\winforms\src\System.Windows.Forms\tests\functests\WinformsControlsTest\WinformsControlsTest.csproj
c:\git\winforms\src\System.Windows.Forms\tests\unittests\System.Windows.Forms.Tests.csproj

Website down

The website is currently down and only displays a "Site Under Construction" title for a blank screen.

SiteDown

This was also raised by a few community members on the C# Discord channel.

Why are build failures ignored during indexing?

<Exec Command="cmd /c &quot;$(PrepareCommand)&quot;" WorkingDirectory="%(ClonedRepository.LocalPath)" ContinueOnError="true" IgnoreStandardErrorWarningFormat="true" LogStandardErrorAsError="false" IgnoreExitCode="true"/>

Currently it seems like build failures for the the repositories to be indexed are ignored. It's weird those are ignored and the build succeeds like nothing happened; this leads to the server running fine but not actually showing the results for the failed repository.

I suppose this is the reason for issues like #60?

cmake is missing from build machines

After #15 we still have 2 repos failing to build. (https://mseng.visualstudio.com/Tools/_build/results?buildId=7742669&view=logs)

It looks like if you can get CMAKE on the build machine, we would get past this and I can look at any remaining issues

CMake is a pre-requisite to build this repository but it was not found on the path. Please install CMake from http://www.cmake.org/download/ and ensure it is on your path.

@alexperovich can you please help with that?

cc @stephentoub

Title on page says ".NET CORE Source"

The title at the top of the page says ".NET Core Source". However, this is no longer true.

Also, as other issues have pointed out, the WinForms and WPF sources are missing.

ASP.NET Core types are no longer available

image

The types for ASP.NET Core are no longer available on the source browser. About one week ago its was still possible to search for ASP.NET Core types.

Were they intentionally removed? And if so is there another source code browser that can be used instead?

Update to 3.1/5.0

I see the site is on netcoreapp2.1, thoughts about updating to 3.1/5.0? Happy to have a crack at this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.