Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subprocesses cannot open input files created by bugpoint #37738

Open
Meinersbur opened this issue Jul 31, 2018 · 10 comments
Open

Subprocesses cannot open input files created by bugpoint #37738

Meinersbur opened this issue Jul 31, 2018 · 10 comments
Labels
bugzilla Issues migrated from bugzilla tools:bugpoint

Comments

@Meinersbur
Copy link
Member

Bugzilla Link 38390
Version trunk
OS Windows NT
CC @hfinkel,@donhinton,@RKSimon,@pcc

Extended Description

Commit r334630 ("LTO: Keep file handles open for memory mapped files.") broke bugpoint on Windows. The launched opt process cannot access the input file created by bugpoint:

$ (cd 'C:\Users\MEINER1\AppData\Local\Temp\reproduce-bruxta1h' &&
'C:\Users\meinersbur\build\llvm\release\bin\bugpoint.exe'
'-opt-command=C:\Users\meinersbur\build\llvm\release\bin\opt.exe'
'C:\Users\MEINER
1\AppData\Local\Temp\reproduce-bruxta1h\pr3821817-precompute.ll'
-scoped-noalias -polly-codegen)
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-b43aded.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-6517e54.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-6575634.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-7dad9ae.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-81165ee.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-018d544.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-e88e95a.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-d37f5e8.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-29843f5.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-26d3c62.bc: error: Could not open input file:
permission denied
C:\Users\meinersbur\build\llvm\release\bin\opt.exe:
bugpoint-input-dc3b1bc.bc: error: Could not open input file:
permission denied
verify failed!

bugpoint-input-b43aded.bc is a strange file. I can see it in explorer,
but when trying to open it, Notepad claims it doesn't exist and offers
to create it (which will then fail).

Also see https://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20180723/573040.html

@donhinton
Copy link
Collaborator

Are you still having problem? I'd check myself, but don't have windows.

@Meinersbur
Copy link
Member Author

Are you still having problem? I'd check myself, but don't have windows.

Positive; just testes with r341740

@donhinton
Copy link
Collaborator

Okay, I think this commit is the actual culprit:

commit b96a87a
Author: Rafael Espindola rafael.espindola@gmail.com
Date: Thu Nov 16 21:53:51 2017 +0000

Convert the last use of sys::fs::createUniqueFile in bugpoint.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318459 91177308-0d34-0410-b5e6-96231b3b80d8

The problem is that if(BitcodeFile.empty()), a TempFile will be created in a local scope, then get deleted before it's used out of scope further down.

@donhinton
Copy link
Collaborator

btw, you can test this by using "-save-temps" and see if that solves the problem.

@Meinersbur
Copy link
Member Author

I don't think that r318459 is the problem. Before submitting this bug, I bisected it to r341740 (meaning it worked before that commit)

However Rafael's createUniqueFile refacturing indeed caused problems with bugpoint on Windows, but it has been fixed. See:

https://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20171120/504298.html

With -save-temps:

$ bugpoint polly-timeout-grtestutils.ll -save-temps -verify -opt-command=C:\Users\meinersbur\build\llvm\release\bin\opt.exe -opt-args -polly-process-unprofitable
Read input file : 'polly-timeout-grtestutils.ll'
*** All input ok
Running selected passes on program to test for crash: C:\Users\meinersbur\build\llvm\release\bin\opt.exe: bugpoint-input-18caf6b.bc: error: Could not open input file: permission denied
Exited with error code '1'

*** Debugging optimizer crash!
Checking to see if these passes crash: -verify: C:\Users\meinersbur\build\llvm\release\bin\opt.exe: bugpoint-input-75e7dbd.bc: error: Could not open input file: permission denied
Exited with error code '1'

*** Found crashing pass: -verify
Emitted bitcode to 'bugpoint-passes.bc'

*** You can reproduce the problem with: opt bugpoint-passes.bc -verify
Checking for crash with changing conditionals to always jump to true: call1.i.noexc call1.i.noexc.split invoke.cont invoke.cont6 for.cond lpad lpad1 invoke.cont10 invoke.cont17 call.i.i.noexc... <33 total>: C:\Users\meinersbur\build\llvm\release\bin\opt.exe: bugpoint-input-6f44ec7.bc: error: Could not open input file: permission denied
verify failed!

@donhinton
Copy link
Collaborator

Okay, that's a different bug, and I submitted a separate patch for it:

https://reviews.llvm.org/D51836

@donhinton
Copy link
Collaborator

According to Microsoft docs,

https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-setfileinformationbyhandle

"... if the application is using FILE_DISPOSITION_INFO with the DeleteFile member set to TRUE, the file would need DELETE access requested in the call to the CreateFile function."

LLVM wraps these windows specific calls in lib/Support/Windows/Path.inc, but only passes DELETE in calls to remove or rename files. However, since TempFile sets FILE_DESPSITION_INFO.DeleteFile = true, subsequent CreateFile calls need to include DELETE access.

I don't have windows, so I can't test this, but here's a small patch that unilaterally adds DELETE to the underlying CreateFile calls that you can use to test if this is the actual problem.

diff --git a/lib/Support/Windows/Path.inc b/lib/Support/Windows/Path.inc
index 57557dbc3da..95c80eb4fc7 100644
--- a/lib/Support/Windows/Path.inc
+++ b/lib/Support/Windows/Path.inc
@@ -1070,7 +1070,7 @@ static std::error_code openNativeFileInternal(const Twine &Name,
SA.bInheritHandle = Inherit;

HANDLE H =

  •  ::CreateFileW(PathUTF16.begin(), Access,
    
  •  ::CreateFileW(PathUTF16.begin(), Access | DELETE,
                   FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, &SA,
                   Disp, Flags, NULL);
    
    if (H == INVALID_HANDLE_VALUE) {

@Meinersbur
Copy link
Member Author

This patch unfortunately did not fix the problem.

@RKSimon
Copy link
Collaborator

RKSimon commented Sep 10, 2018

*** Bug llvm/llvm-bugzilla-archive#38467 has been marked as a duplicate of this bug. ***

@Meinersbur
Copy link
Member Author

mentioned in issue llvm/llvm-bugzilla-archive#38467

@llvmbot llvmbot transferred this issue from llvm/llvm-bugzilla-archive Dec 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugzilla Issues migrated from bugzilla tools:bugpoint
Projects
None yet
Development

No branches or pull requests

3 participants