AndyGerlicher commented on Apr 1, 2015

MSBuild team triage: This isn't a change we're looking to take anytime soon as it would be fairly substantial with implications on dependent tasks and tools.

ariccio commented on Apr 2, 2015

[...]it would be fairly substantial with implications on dependent tasks and tools.

Could you be a bit more specific?

ariccio commented on Apr 8, 2015

I'm still curious as to what the substantial implications of an incremental fix for MSBuild in isolation are, exactly.

In general, you'd have to have all of the other things you're using understand long paths. There's been a number of discussions internally and externally about the MAX_PATH issue and they always eventually boil down to: "If we could rewrite everything we'd be good but we can't so there's an infinite sea of compat risk here :-(".

So... maybe some day?

Have you looked at AlphaFS?

The key issue here is that it doesn't really matter if MSBUILD is updated. It's not an island. it passes paths to lots of other things (OS apis, tools etc) and updating yourself in a way that doesn't break them is a Hard Problem (tm). Almost certainly not impossible but there's lots of other work with better payoffs.

ariccio commented on Apr 10, 2015

Hmm. The way I see it, is that the entire toolchain is not compatible with paths longer than MAX_PATH, and thus by (incrementally) fixing a single component will not lead to toolchain-wide regression. It'll just be broken in a different place - the entire toolchain, when used as a system, will still be broken, but in a slightly smaller way.

It's even worse, by the way, as MAX_PATH was only the "maximum total file path length" for FAT16, and pre-Unicode/pre-NT APIs. Any documentation that says MAX_PATH is the longest possible file path is horribly wrong! MAX_PATH should really have been named MAX_FAT_PATH or MAX_PATH_COMPONENT; any program that was built after Windows NT 3.1 (yes, July 27, 1993, the earliest default use of NTFS that I know of) to use Unicode APIs with the assumption that MAX_PATH is the longest possible file path is, and always was, terminally broken. GCC 4.9-style broken.

I can even find discussions dating more than ten years back, of developers realizing that assuming MAX_PATH is the maximum length of a path is an outdated (ridiculous) assumption.

Any components that have been retroactively "fixed" to ensure compatibility with only paths that are MAX_PATH or shorter - well, that's just crazy.

Also, I can't imagine that we can break the OS APIs themselves by using them in a documented manner.

So yeah, it's a Hard Problem™, and I know that we don't yet have proper time travel, but if we attack it incrementally, we either make small (and effectively non-functional) improvements, or expose bugs/unforeseen design issues in other components.

I say effectively non-functional, because those components that can't handle MAX_PATH are already broken for paths longer than MAX_PATH. I can't imagine that those applications rely internally on the exact full path name that's given to them.

Furthermore, I imagine that OTHER toolchains use MSBuild in ways that are crippled by the MAX_PATH limitations.

...and while new features/better compliance are nice, I think they're counterproductive/add silly complexity if the core components that they're built on are shaky and fail in unexpected ways (e.g. MAX_PATH limitations). Putting it off into the future only makes it harder to fix.

Lastly, let me remind you of the nearly 3000 users who voted on "Fix 260 character file name length limitation" before it was declined. There are still users commenting on the issue with frustration.

Here's an idea

How about you/we go ahead and make the changes to MSBuild, make it available as a "preview" or "technology demonstration", and see what (if anything) breaks.

You seem to understand the issue pretty well. I don't disagree that isn't a frustrating experience some times but they have finite resources and can only take so much risk. I'd be more interested in them making sure cross plat is brought up cleanly but it's up to them. I'm sure they appreciate your enthusiasm and feedback. I'll bow out of this issue for now.. lots of other bugs to track down.

ariccio commented on Apr 10, 2015

Fair enough.

@ariccio If you have the inclination, I say go for it. Everybody will eventually benefit, especially if more pieces of the toolchain go open source. Saying that it can't be fixed because everything else in every possible toolchain would also have to be fixed immediately is, quite simply, a non-starter and completely unrealistic. The whole point of going open source is to let people scratch their itches, especially when Microsoft doesn't have the resources to do it themselves...

One way to minimize the risk associated with backwards and toolchain compatibility that Microsoft keeps talking about is to ensure that the current limitation remains the default behavior while making it possible to opt-in to longer paths. The opt-in mechanism can be via any relevant configuration mechanism, including a new command line option, a build file setting, a registry setting and/or some other mechanism.

I'm also in.

Especially down the road to xplat, where *NIX operating systems do not care about a 260 char path length at all. IF set at all (this may well be -1 to indicate that there is no limit) it for example defaults to 4096 on x86 Linux. On OS X's HFSP the max path length is unlimited (while the file name is limited to 255).

So, in fact, this issue here is a blocker for xplat.

Yes, there may be other places in VS, TFS and other tools that have problems with this, but as long as we want to simply 'MSBuild mysolution' on commandline or on an CI system like AppVoyer or Travis, and happen to have paths longer than 260 chars there, it should not artificially prevent us from doing so.

I also don't see how removing this limitation would break other stuff.
If other stuff can't handle paths longer than 260 chars, it already IS broken. It won't break more. I would just break elsewhere.

And: Yes. Especially because MSBuild is not an island, but a part in a chain, someone has to start. And MSBuild is the perfect place because it can be used more or less standalone on new DNX projects (because, honestly, Sake is the totally wrong thing for building stuff), and it could be the door-opener to fix the stuff in other places too. Best thing is, that others like the VS or TFS team won't have excuses like 'MSBuild doesn't support that, so why should we change that?' anymore.

When talking about where to spent time, it's mostly the time of the community that will be spend here. So why not simply start here, by removing that artifical limitation, pass longer paths around where necessary and, who knows, maybe much less is broken that is currently being thought off and a lot of stuff simply starts to work out of the box?

Piedone commented on May 29, 2015

Mediaeval issues like this still prevent us from using proper folder structures and proper file/folder names still. And it's not a corner case: for us all of our builds would fail if not copied to the drive root of the build server!

hacst commented on Sep 4, 2015

This is something everyone with a complex build in a build system encounters sooner or later and it is quite annoying to work around. Fixing it has to start somewhere.

ariccio commented on Sep 4, 2015

Glad to see that people are still voicing their support. Fixing this arcane issue is certainly not a glamorous job - hence the foot-dragging - but it's a desperately important one.

I'd also see this open 10 years from now if noone does a start!

But we also need a Api for full length paths directly in dotnet core!

dsplaisted commented on Sep 7, 2015

Work is now in progress to enable long path support for .NET Core. I think the goal is to also add this to the full .NET Framework (@terrajobst can probably confirm whether this is the case).

Once this is done it should be a lot easier to add support for long paths to MSBuild- long paths should "just work" when they are passed to .NET APIs, without having to add the \\?\ prefix.

@AndyGerlicher Should we consider re-opening this issue?

ariccio commented on Sep 7, 2015

Should we consider re-opening this issue?

Technically, it is still open.

But, if long path support is on it's way to the .NET Core, then I'd be totally cool treating this as a tracking issue.

Also running in to this same issue (with ASP.NET 5 DNX Beta 7 website publish operation):
System.ArgumentException: The name can be no more than 260 characters in length.

Do you have an update on when this will be resolved?

Thanks!

AndyGerlicher commented on Oct 15, 2015

I know there's been some progress getting long path support in the core clr, but I think it will be a while before it's ported back to the desktop framework. Until that happens, this just isn't really high on our priority list. It's something we would probably do in xplat first since we'll get the core clr updates sooner, but it's not on our radar right now.

terrajobst commented on Oct 15, 2015

Given how far we are with making progress on solving it in .NET Core, I wouldn't invest in handling it in MSBuild. At least for the .NET Core based version it would get a free ride by doing nothing. .NET Framework is a bit more out, but again, our goal is to port those changes back and make it transparent which would mean that MSBuild would get a free ride.

Either way, I don't think doing a duct tape fix in MSBuild is currently worth it.

AndyGerlicher commented on Oct 15, 2015

Completely agree. However the one caveat is we have several places in our code where we check if the path is over some const defined in our code. So we won't quite get a free ride.

terrajobst commented on Oct 15, 2015

If that's the case, I'd encourage you to remove those. This isn't even a long path thing; for cross-platform alone you don't want to restrict paths on Mac/Linux to a Windows specific limit. In other words, removing those checks immediately benefits some customers.

(Also, on Windows you don't really lose anything. You might trade errors against different errors or run for longer before detecting it but that seems like the correct behavior anyways moving forward.)

ariccio commented on Oct 16, 2015

Given how far we are with making progress on solving it in .NET Core, I wouldn't invest in handling it in MSBuild.

I'm glad to hear that that project is progressing nicely! Any idea on how long "a bit more out" is?

Of course, it's software, so I wont hold it against you as a "deadline" of some sort :)

terrajobst commented on Oct 19, 2015

We don't have a timeline but I'll keep you posted via our design reviews, Twitter and so on. Hopefully the dev driving it can give us an update in a couple of weeks.

rainersigwald commented on Jan 31, 2019

MAX_PATH in MSBuild 16.0

tl;dr: Install Visual Studio 2019 Preview 2, enable long paths, and things are slightly better.

What's changed

As of Visual Studio 2019 Preview 2 (which includes MSBuild 16.0.360-preview), MSBuild.exe now (as of #3507) opts into support for long paths.

This is dependent on .NET Framework, operating system, and host-executable support, so there is a big prerequisite: you must enable long paths in Windows by using Group Policy or setting a registry key. There's Windows documentation about the registry key.

Does that mean things work?

No, for all the reasons detailed in comments above: just because MSBuild works, doesn't mean your build will, because many other tools are involved.

devenv.exe, the main Visual Studio process, does not yet opt into support. That means only command-line builds will be affected by the MSBuild changes.

Where should I follow up?

If you encounter a long-path problem with a specific executable, tool, or task, ideally report it to the owner of that tool. For instance, I filed dotnet/roslyn#32804 because the C# and VB compilers need to make similar changes to what MSBuild did.

If the task is shipped from this repo, please file a new issue in this repo.

The overall "Visual Studio should support long paths" item is https://developercommunity.visualstudio.com/idea/351628/allow-building-running-and-debugging-a-net-applica.html.

rainersigwald commented on Feb 7, 2019

I'm going to close this as "done as much as we can do for 16.0". See the comment above for the many caveats involved, and what to do if you find a specific MAX_PATH problem.

terrajobst commented on Feb 7, 2019

Agreed. We're clearly interested in addressing the issue, but we can't drive issues that are stated that broadly.

dsplaisted commented on Feb 7, 2019

@rainersigwald @KathleenDollard @mairaw Can we include the info from Rainer's comment in the Dev16 release notes and documentation?

tl;dr: Install Visual Studio 2019 Preview 2, enable long paths, and things are slightly better

Is there a standalone MSBuild Tools equivalent to this? I've been able to avoid installing VS on my 10+ build agents so far and I don't plan on starting now with a preview release just to fix this silly problem.

@rainersigwald was this expected to be fixed in both the 32bit and 64bit (amd64\msbuild.exe) msbuild binaries? I am getting interesting behavior wherein the 64 bit version throws this error:

"S:\Builds\GenerateInterop-60R39\WorkingDirectory\src\Dotnet\Source\Framework\Interop\Project\ComputersUnlimited.Interop.sln" (Restore target) (1) ->
(Restore target) ->
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(121,5): error : The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters. [S:\Builds\GenerateInterop-60R39\WorkingDirectory\src\Dotnet\Source\Framework\Interop\Project\ComputersUnlimited.Interop.sln]

Whereas using the 32bit version has no such qualms (restores and builds without issue).

Here's the version from the 64bit and 32bit versions (same):

Microsoft (R) Build Engine version 16.1.76+g14b0a930a7 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

I can open a new issue if need be but wanted to start here. If this is something I need to get off to the NuGet guys I can do that as well.

mahaase commented on Dec 17, 2019 •

Has anyone ever seen/verified, that this fix still works?

I enabled long paths in gpedit.
I checked msbuild.exe resource stuff, still fine.
I use MSBuild v16.2, but the long path issues still exists.

"...maximale Pfadlimit des Betriebssystems. Der vollqualifizierte Dateiname muss weniger als 260 Zeichen umfassen"

The issue occurs inside a build-system, which handles the paths (directory-names a.s.o.) by itself, no way/easy way to change them.

inf9144 commented on Apr 3

@mahaase same problem here. 32bit is still working, 64bit does show the max_path error. Tested with newest visual studio build tools on newest windows 10. Thanks for the tip with 32bit. I was nearly giving up and taking myself to a knit.

inf9144 commented on Apr 8

@rainersigwald could you please reopen this ticket because of the bug in 64bit?

KirillOsenkov commented on Apr 8

I think this should be a new issue, with specifics about 64-bit not working as expected

mahaase commented on May 4

I think this should be a new issue, with specifics about 64-bit not working as expected

I opened a new ticket for 64-bit variant. I hope the issue will solved soon, now. This bug is a show-stopper.

Piedone commented on May 4

Worth linking to it: #5331

brignolij commented on Oct 14 •

Hi, I'm facing the issue with path limit using AWS codebuild (meaning buildspec.yml file) , msbuild and with docker image (mcr.microsoft.com/dotnet/framework/sdk, https://hub.docker.com/_/microsoft-dotnet-framework-sdk).
Any suggestion to make it work using only powershell/cmd command ?
more documentation :
https://aws.amazon.com/fr/blogs/devops/creating-ci-cd-pipelines-for-asp-net-4-x-with-aws-codepipeline-and-aws-elastic-beanstalk/

rainersigwald commented on Oct 14

@brignolij I'm sorry, I don't know whether the container hosts need the long-path configuration, or just the container images. If setting the registry key inside the container doesn't work, I think you'd have to ask AWS support.

brignolij commented on Oct 15 •

@rainersigwald Thanks for your answer.
I already have a issue open with AWS support, they suggest me to check on msbuild support. I have a project with long path. (lot of files ,folders). So i really want to make it work with long path.
i test this following command into my buildsepc
Set-ItemProperty 'HKLM:\System\CurrentControlSet\Control\FileSystem' -Name 'LongPathsEnabled' -value 1
but this command requiere a system restart to be effective and this not possible on build system like awscodebuild.
I'm open to any suggestion.
Also is application manifest file required for .net framework mvc app?
is yes where should we add ?

<application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
</application>

Regards