EDN Admin
Well-known member
<p style=" Hi,
<p style=" I am seeing a strange problem in my project where perl could not see a file though it is present in the disk. We run a series of short backend jobs ( each spanning 10 s ) through perl. The backend job write an output file and exit, later which
perl process would try to transfer it. The perl process is threaded where master thread acts as listener waiting for connection/messages and send file list to child thread ( by checking for file existence ) about what files to transfer. And child
thread does the actual file transfer.
<p style=" The job runs fine initially and all of a sudden master thread fail to detect file written by the backend. Debugging perl code ( 5.10.1 from http://www.cpan.org/src/ http://www.cpan.org/src/ ), I found stati64 of
CRT fail and return -1. On retry, the call seem to work fine. I can guarantee there there is no race condition involved by backend process as we try to access the file in perl only after backend exit. I also tried to capture the events using ProcMon enabling
File System Activity and made sure there are no external process accessing the file. When the failure happens with stati64, I see the following events in procmon -
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px Time of Day<span style="white-spacere
Process Name<span style="white-spacere PID<span style="white-spacere
TID<span style="white-spacere Operation<span style="white-spacere
Path<span style="white-spacere Result<span style="white-spacere
Detail<span style="white-spacere Description
<p style="
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5315529 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
3496<span style="white-spacere CreateFile<span style="white-spacere
D:scrrajagopatest_31test_31.01.mae<span style="white-spacere
SUCCESS<span style="white-spacere Desired Access: Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: None, AllocationSize: n/a, OpenResult: Opened<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5315965 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
1636<span style="white-spacere DeviceIoControl<span style="white-spacere
D:scrrajagopatest_31nyc-bld-w06-0-4eb305ae<span style="white-spacere
INVALID PARAMETER<span style="white-spacere Control: 0x12043 (Device:0x1 Function:2064 Method: 3)<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5316010 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
3496<span style="white-spacere QueryInformationVolume<span style="white-spacere
D:scrrajagopatest_31test_31.01.mae<span style="white-spacere
SUCCESS<span style="white-spacere VolumeCreationTime: 1/20/2011 7:34:15 PM, VolumeSerialNumber: 188D-F175, SupportsObjects: True, VolumeLabel: scr<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5316301 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
3496<span style="white-spacere QueryAllInformationFile<span style="white-spacere
D:scrrajagopatest_31test_31.01.mae<span style="white-spacere
BUFFER OVERFLOW<span style="white-spacere CreationTime: 11/3/2011 5:20:47 PM, LastAccessTime: 11/3/2011 5:20:47 PM, LastWriteTime: 11/3/2011 5:20:47 PM, ChangeTime: 11/3/2011 5:20:47 PM, FileAttributes: A, AllocationSize: 128, EndOfFile: 126, NumberOfLinks:
1, DeletePending: False, Directory: False, IndexNumber: 0x8e00000002f208, EaSize: 0, Access: Read Attributes, Synchronize, Position: 0, Mode: Synchronous IO Non-Alert, AlignmentRequirement: Word<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5316368 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
1636<span style="white-spacere DeviceIoControl<span style="white-spacere
D:scrrajagopatest_31nyc-bld-w06-0-4eb305ae<span style="white-spacere
INVALID PARAMETER<span style="white-spacere Control: 0x12043 (Device:0x1 Function:2064 Method: 3)<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5316622 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
3496<span style="white-spacere CloseFile<span style="white-spacere
D:scrrajagopatest_31test_31.01.mae<span style="white-spacere
SUCCESS<span style="white-spacere n/a<span style="white-spacere
NYCrajagopa
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
As you can see, the set of file operations involved with "test_31.01.mae" using stati64 is shown as SUCCESS. But the result of stati64(..) is -1 and GetLastError(..) give 6. Why would this be? Is procmon showing wrong status? Is stati64 buggy in threaded
environment? Is it safe to retry the stati64 call if return value is -1 and GetLastError() is 6? Please throw pointers as this issue is important to fix soon.
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
-Karthik
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
View the full article
<p style=" I am seeing a strange problem in my project where perl could not see a file though it is present in the disk. We run a series of short backend jobs ( each spanning 10 s ) through perl. The backend job write an output file and exit, later which
perl process would try to transfer it. The perl process is threaded where master thread acts as listener waiting for connection/messages and send file list to child thread ( by checking for file existence ) about what files to transfer. And child
thread does the actual file transfer.
<p style=" The job runs fine initially and all of a sudden master thread fail to detect file written by the backend. Debugging perl code ( 5.10.1 from http://www.cpan.org/src/ http://www.cpan.org/src/ ), I found stati64 of
CRT fail and return -1. On retry, the call seem to work fine. I can guarantee there there is no race condition involved by backend process as we try to access the file in perl only after backend exit. I also tried to capture the events using ProcMon enabling
File System Activity and made sure there are no external process accessing the file. When the failure happens with stati64, I see the following events in procmon -
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px Time of Day<span style="white-spacere
Process Name<span style="white-spacere PID<span style="white-spacere
TID<span style="white-spacere Operation<span style="white-spacere
Path<span style="white-spacere Result<span style="white-spacere
Detail<span style="white-spacere Description
<p style="
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5315529 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
3496<span style="white-spacere CreateFile<span style="white-spacere
D:scrrajagopatest_31test_31.01.mae<span style="white-spacere
SUCCESS<span style="white-spacere Desired Access: Read Attributes, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: None, AllocationSize: n/a, OpenResult: Opened<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5315965 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
1636<span style="white-spacere DeviceIoControl<span style="white-spacere
D:scrrajagopatest_31nyc-bld-w06-0-4eb305ae<span style="white-spacere
INVALID PARAMETER<span style="white-spacere Control: 0x12043 (Device:0x1 Function:2064 Method: 3)<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5316010 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
3496<span style="white-spacere QueryInformationVolume<span style="white-spacere
D:scrrajagopatest_31test_31.01.mae<span style="white-spacere
SUCCESS<span style="white-spacere VolumeCreationTime: 1/20/2011 7:34:15 PM, VolumeSerialNumber: 188D-F175, SupportsObjects: True, VolumeLabel: scr<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5316301 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
3496<span style="white-spacere QueryAllInformationFile<span style="white-spacere
D:scrrajagopatest_31test_31.01.mae<span style="white-spacere
BUFFER OVERFLOW<span style="white-spacere CreationTime: 11/3/2011 5:20:47 PM, LastAccessTime: 11/3/2011 5:20:47 PM, LastWriteTime: 11/3/2011 5:20:47 PM, ChangeTime: 11/3/2011 5:20:47 PM, FileAttributes: A, AllocationSize: 128, EndOfFile: 126, NumberOfLinks:
1, DeletePending: False, Directory: False, IndexNumber: 0x8e00000002f208, EaSize: 0, Access: Read Attributes, Synchronize, Position: 0, Mode: Synchronous IO Non-Alert, AlignmentRequirement: Word<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5316368 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
1636<span style="white-spacere DeviceIoControl<span style="white-spacere
D:scrrajagopatest_31nyc-bld-w06-0-4eb305ae<span style="white-spacere
INVALID PARAMETER<span style="white-spacere Control: 0x12043 (Device:0x1 Function:2064 Method: 3)<span style="white-spacere
n/a<span style="white-spacere NYCrajagopa
<p style=" <span style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif <span style="font-size:14px; line-height:18px 5:20:59.5316622 PM<span style="white-spacere
perl.exe<span style="white-spacere 3444<span style="white-spacere
3496<span style="white-spacere CloseFile<span style="white-spacere
D:scrrajagopatest_31test_31.01.mae<span style="white-spacere
SUCCESS<span style="white-spacere n/a<span style="white-spacere
NYCrajagopa
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
As you can see, the set of file operations involved with "test_31.01.mae" using stati64 is shown as SUCCESS. But the result of stati64(..) is -1 and GetLastError(..) give 6. Why would this be? Is procmon showing wrong status? Is stati64 buggy in threaded
environment? Is it safe to retry the stati64 call if return value is -1 and GetLastError() is 6? Please throw pointers as this issue is important to fix soon.
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
-Karthik
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
<div style="font-family:Arial,Liberation Sans,DejaVu Sans,sans-serif; font-size:14px; line-height:18px
View the full article