boostgsoc13 / boost.afio Goto Github PK

This was working, but I forgot to propagate the changes to the new unit test infrastructure.

common.mkdir ......\bin.v2\libs\afio\test\async_io_torture_direct_sync_test.test\gcc-mingw-4.7.2
common.mkdir ......\bin.v2\libs\afio\test\async_io_torture_direct_sync_test.test\gcc-mingw-4.7.2\debug
common.mkdir ......\bin.v2\libs\afio\test\async_io_torture_direct_sync_test.test\gcc-mingw-4.7.2\debug\threading-multi
gcc.compile.c++ ......\bin.v2\libs\afio\test\async_io_torture_direct_sync_test.test\gcc-mingw-4.7.2\debug\threading-multi\async_io_torture_direct_sync_test.o
In file included from ....../boost/asio/detail/win_iocp_handle_service.hpp:319:0,
from ....../boost/asio/detail/win_iocp_serial_port_service.hpp:26,
from ....../boost/asio/serial_port_service.hpp:27,
from ....../boost/asio/basic_serial_port.hpp:30,
from ....../boost/asio.hpp:26,
from ../../../boost/afio/afio.hpp:33,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/asio/detail/impl/win_iocp_handle_service.ipp: In constructor 'boost::asio::detail::win_iocp_handle_service::overlapped_wrapper::overlapped_wrapper(boost::system::error_code&)':
....../boost/asio/detail/impl/win_iocp_handle_service.ipp:49:44: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
In file included from ....../boost/test/utils/runtime/cla/dual_name_parameter.hpp:19:0,
from ....../boost/test/impl/unit_test_parameters.ipp:31,
from ....../boost/test/included/unit_test.hpp:33,
from test_functions.hpp:29,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/test/utils/runtime/config.hpp: In function 'void boost::runtime::putenv_impl(boost::runtime::cstring, boost::runtime::cstring)':
....../boost/test/utils/runtime/config.hpp:95:51: error: 'putenv' was not declared in this scope
In file included from async_io_torture_direct_sync_test.cpp:1:0:
test_functions.hpp: In function 'void _1000_open_write_close_deletes(std::shared_ptrboost::afio::async_file_io_dispatcher_base, size_t)':
test_functions.hpp:80:76: error: 'to_string' is not a member of 'std'
test_functions.hpp:80:76: note: suggested alternatives:
In file included from ....../boost/exception_ptr.hpp:9:0,
from ....../boost/thread/future.hpp:28,
from ../../../boost/afio/afio.hpp:35,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/exception/detail/exception_ptr.hpp:487:5: note: 'boost::to_string'
In file included from ....../boost/exception/to_string_stub.hpp:15:0,
from ....../boost/exception/info.hpp:16,
from ....../boost/exception/detail/exception_ptr.hpp:20,
from ....../boost/exception_ptr.hpp:9,
from ....../boost/thread/future.hpp:28,
from ../../../boost/afio/afio.hpp:35,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/exception/to_string.hpp:72:5: note: 'boost::to_string'
In file included from async_io_torture_direct_sync_test.cpp:1:0:
test_functions.hpp: In function 'void evil_random_io(std::shared_ptrboost::afio::async_file_io_dispatcher_base, size_t, size_t, size_t)':
test_functions.hpp:317:72: error: 'to_string' is not a member of 'std'
test_functions.hpp:317:72: note: suggested alternatives:
In file included from ....../boost/exception_ptr.hpp:9:0,
from ....../boost/thread/future.hpp:28,
from ../../../boost/afio/afio.hpp:35,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/exception/detail/exception_ptr.hpp:487:5: note: 'boost::to_string'
In file included from ....../boost/exception/to_string_stub.hpp:15:0,
from ....../boost/exception/info.hpp:16,
from ....../boost/exception/detail/exception_ptr.hpp:20,
from ....../boost/exception_ptr.hpp:9,
from ....../boost/thread/future.hpp:28,
from ../../../boost/afio/afio.hpp:35,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/exception/to_string.hpp:72:5: note: 'boost::to_string'
In file included from async_io_torture_direct_sync_test.cpp:1:0:
test_functions.hpp:426:136: error: 'to_string' was not declared in this scope
test_functions.hpp:426:136: note: suggested alternatives:
In file included from ....../boost/exception_ptr.hpp:9:0,
from ....../boost/thread/future.hpp:28,
from ../../../boost/afio/afio.hpp:35,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/exception/detail/exception_ptr.hpp:487:5: note: 'boost::to_string'
In file included from ....../boost/exception/to_string_stub.hpp:15:0,
from ....../boost/exception/info.hpp:16,
from ....../boost/exception/detail/exception_ptr.hpp:20,
from ....../boost/exception_ptr.hpp:9,
from ....../boost/thread/future.hpp:28,
from ../../../boost/afio/afio.hpp:35,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/exception/to_string.hpp:72:5: note: 'boost::to_string'
In file included from async_io_torture_direct_sync_test.cpp:1:0:
test_functions.hpp:451:55: error: 'to_string' was not declared in this scope
test_functions.hpp:451:55: note: suggested alternatives:
In file included from ....../boost/exception_ptr.hpp:9:0,
from ....../boost/thread/future.hpp:28,
from ../../../boost/afio/afio.hpp:35,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/exception/detail/exception_ptr.hpp:487:5: note: 'boost::to_string'
In file included from ....../boost/exception/to_string_stub.hpp:15:0,
from ....../boost/exception/info.hpp:16,
from ....../boost/exception/detail/exception_ptr.hpp:20,
from ....../boost/exception_ptr.hpp:9,
from ....../boost/thread/future.hpp:28,
from ../../../boost/afio/afio.hpp:35,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/exception/to_string.hpp:72:5: note: 'boost::to_string'
In file included from ....../boost/thread/shared_mutex.hpp:18:0,
from ....../boost/thread/detail/thread_group.hpp:9,
from ....../boost/thread/thread.hpp:13,
from ../../../boost/afio/afio.hpp:34,
from test_functions.hpp:18,
from async_io_torture_direct_sync_test.cpp:1:
....../boost/thread/win32/shared_mutex.hpp: In instantiation of 'T boost::shared_mutex::interlocked_compare_exchange(T*, T, T) [with T = boost::shared_mutex::state_data]':
....../boost/thread/win32/shared_mutex.hpp:124:103: required from here
....../boost/thread/win32/shared_mutex.hpp:52:99: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
....../boost/thread/win32/shared_mutex.hpp:52:99: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
....../boost/thread/win32/shared_mutex.hpp:53:52: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
....../boost/thread/win32/shared_mutex.hpp:53:52: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]

afio_iocp.ipp 1385

            do
            {
                ...
                BOOL ok=DeviceIoControl(p->native_handle(), FSCTL_QUERY_ALLOCATED_RANGES, buffers->data(), sizeof(FILE_ALLOCATED_RANGE_BUFFER), buffers->data(), (DWORD)(buffers->size()*sizeof(FILE_ALLOCATED_RANGE_BUFFER)), &bytesout, ol.get());
                DWORD errcode=GetLastError();
                if(!ok && ERROR_IO_PENDING!=errcode)
                {
                  if(ERROR_INSUFFICIENT_BUFFER==errcode || ERROR_MORE_DATA==errcode)
                  {
                    buffers->resize(buffers->size()*2);
                    continue;   // < ==========
                  }
                  //std::cerr << "ERROR " << errcode << std::endl;
                  asio::error_code ec(errcode, asio::error::get_system_category());
                  ol.complete(ec, ol.get()->InternalHigh);
                }
                else
                  ol.release();
            } while(false); // < ==========

"continue" will terminate the loop instead of continuing it.

This is due to a bug in the WOW64 syscall translation layer. Microsoft have told me they won't fix it as I'm not supposed to be using NT kernel APIs directly, so I'll need to figure out a workaround.

Specifically:

1. Microsoft Visual Studio 2010
2. GNU Compiler Collection v4.6
3. clang v3.1

clang-linux.link bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o: In function boost::detail::lexical_cast_do_cast<int, boost::unit_test::basic_cstring<char const> >::lexical_cast_impl(boost::unit_test::basic_cstring<char const> const&)': /home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/lexical_cast.hpp:2351: undefined reference totypeinfo for int'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o: In function boost::detail::lexical_cast_do_cast<long, boost::unit_test::basic_cstring<char const> >::lexical_cast_impl(boost::unit_test::basic_cstring<char const> const&)': /home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/lexical_cast.hpp:2351: undefined reference totypeinfo for long'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o: In function boost::detail::lexical_cast_do_cast<bool, boost::unit_test::basic_cstring<char const> >::lexical_cast_impl(boost::unit_test::basic_cstring<char const> const&)': /home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/lexical_cast.hpp:2351: undefined reference totypeinfo for bool'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o: In function boost::detail::lexical_cast_do_cast<unsigned int, boost::unit_test::basic_cstring<char const> >::lexical_cast_impl(boost::unit_test::basic_cstring<char const> const&)': /home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/lexical_cast.hpp:2351: undefined reference totypeinfo for unsigned int'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o: In function boost::exception const* boost::exception_detail::get_boost_exception<std::domain_error>(std::domain_error const*)': /home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/exception/detail/exception_ptr.hpp:257: undefined reference to__dynamic_cast'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o: In function boost::exception const* boost::exception_detail::get_boost_exception<std::invalid_argument>(std::invalid_argument const*)': /home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/exception/detail/exception_ptr.hpp:257: undefined reference to__dynamic_cast'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o: In function boost::exception const* boost::exception_detail::get_boost_exception<std::length_error>(std::length_error const*)': /home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/exception/detail/exception_ptr.hpp:257: undefined reference to__dynamic_cast'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o: In function boost::exception const* boost::exception_detail::get_boost_exception<std::out_of_range>(std::out_of_range const*)': /home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/exception/detail/exception_ptr.hpp:257: undefined reference to__dynamic_cast'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o: In function boost::exception const* boost::exception_detail::get_boost_exception<std::logic_error>(std::logic_error const*)': /home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/exception/detail/exception_ptr.hpp:257: undefined reference to__dynamic_cast'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o:/home/jenkins-slave/workspace/Boost.AFIO Build Linux clang 3.3 + libc++/boost-trunk/./boost/exception/detail/exception_ptr.hpp:257: more undefined references to __dynamic_cast' follow bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o:(.data.rel.ro._ZTIN5boost9unit_test12report_levelE[_ZTIN5boost9unit_test12report_levelE]+0x0): undefined reference tovtable for __cxxabiv1::__enum_type_info'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o:(.data.rel.ro._ZTIN5boost9unit_test9log_levelE[_ZTIN5boost9unit_test9log_levelE]+0x0): undefined reference to vtable for __cxxabiv1::__enum_type_info' bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o:(.data.rel.ro._ZTIN5boost9unit_test13output_formatE[_ZTIN5boost9unit_test13output_formatE]+0x0): undefined reference tovtable for __cxxabiv1::__enum_type_info'
bin.v2/libs/afio/test/async_io_works_64_sync_test.test/clang-linux-3.3/debug/threading-multi/async_io_works_64_sync_test.o:(.data.rel+0x40): undefined reference to `typeinfo for char const*'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Right now, the unit and functional testing mostly tests success. For any library which can cause data loss like this one, very robust error handling functional testing is needing. I'm think that as an absolute minimum we need to mount a filing system via a looped back file and intentionally test out of disc space error handling on it for correctness.

Other idea is to deliberately bitflip corrupt the loopback file to test how we handle weird filing system errors.

I have already mapped in the code examples per API in the documentation, but they are currently unimplemented. I'll also need the examples added to the unit test suite, might as well kill two birds with one stone.

Straight from /analyze

C6011   Dereferencing NULL pointer 'q'.
        40 Enter this branch, (assume 'first_')
        42 Skip this branch, (assume 'q==first_' is false)
        49 Enter this loop, (assume 'p->next_')
        51 Enter this branch, (assume 'p->next_==q')
        53 'q' is dereferenced, but may still be NULL

Does your CI run /analyze? ( I'm not familiar with CI )

Right now on Linux we keep an open handle to the containing directory of each open file because ext2 and some mount flags of ext3 and ext4 require you to fsync the directory every time you add or delete file entries. Such an open file handle makes enumerating that directory super quick, so might as well do so.

Note that using the kernel syscall directly is preferred - glibc's readdir() implementation is notoriously unable to scale well.

We should also add a unit test where one creates a million files, and tries enumerating them. That normally murders Linux, but our direct syscall implementation ought not to suffer from that.

See http://undocumented.ntinternals.net/UserMode/Undocumented%20Functions/NT%20Objects/File/NtSetInformationFile.html

I must admit, I haven't thought of a suitable quick start mini-project yet. Maybe a storage benchmarking tool or something? Maybe a benchmarker of filing system caching behaviour? I don't know.

Failed hard drive replacement is happening as I type this.

In afio.ipp, at line 33, the definition of BOOST_AFIO_DEBUG_PRINT is controlled by a(n always false) conditional.

Rather than hacking away at header files (which I usually set to read-only), perhaps another #define, something like ENABLE_BOOST_AFIO_DEBUG_PRINT would be better?

i.e. #if defined(DEBUG) && defined(ENABLE_BOOST_AFIO_DEBUG_PRINT)

Keep up the great work!

BTRFS on Linux and ZFS on Linux both provide very fast constant time filing system snapshotting. Snapshots can be used to avoid needing to use O_SYNC or fsync to enforce data writing order with really fast performance, and therefore we ought to support those. For Windows, it might be worth investigating Transactional NTFS, though I believe that is supposed to be deprecated now.

Need to rewrite the appropriate parts of Boost.ASIO to fix this problem seeing as its maintainer has left the bug report languish.

Make sure it will not overwrite one of these pseudo symlinks. Might want to add a flag to stat_t about pseudo-symlinks too.

Then the afio::future::then_XXX expands into:

afio::future::then([args...](afio::future f){ return f->XXX(args...);})

In other words, when this future signals, schedule some other operation on the newly ready future.

The Hello World example has a bug in the try-block:

boost::afio::async_path_op_req req("example_file.txt",
                                   boost::afio::file_flags::ReadWrite);

Another file flag is needed for correct operation: file_flags::Create or maybe file_flags::CreateOnlyIfNotExists. Otherwise the file is never created and the rest of the example is garbage.

This may very well be a false positive from /analyze, but I can't rule out some subtle bug.
/analyze says, in very many words:

C6001   Using uninitialized memory 'buffer'
        75 'buffer' is not initialized
        84 Skip this branch, (assume '<branch condition>' is false)
        93 Skip this branch, (assume 'needInternal' is false)
        100 Skip this branch, (assume 'needBasic' is false)
        107 Skip this branch, (assume 'needStandard' is false)
        115 Skip this branch, (assume '<branch condition>' is false)
        128 Skip this branch, (assume '!!((&(wanted, ino)))' is false)
        129 Skip this branch, (assume '!!((&(wanted, type)))' is false)
        130 Skip this branch, (assume '!!((&(wanted, nlink)))' is false)
        131 Skip this branch, (assume '!!((&(wanted, atim)))' is false)
        132 Skip this branch, (assume '!!((&(wanted, mtim)))' is false)
        133 Skip this branch, (assume '!!((&(wanted, ctim)))' is false) 
        138 Enter this branch, (assume '!!((&(wanted, birthtim)))')
        138 'buffer' is used, but may not have been initialized

This very well might be just /analyze not properly handling the unusual branch condition at 84 (if((needInternal+needBasic+needStandard)>=2)) - as that line generates a C6323- but direntry is complex enough that I figured that I should bring it to your attention nonetheless.

In afio.ipp, at 2116, std::filesystem::path::value_type buffer is initialized with the statement:

BOOST_AFIO_TYPEALIGNMENT(8) std::filesystem::path::value_type buffer[sizeof(FILE_ALL_INFORMATION)/sizeof(std::filesystem::path::value_type)+32769];

And then on 2122, it's cast to a (FILE_ID_FULL_DIR_INFORMATION *). Is this intended?

A static_assert( sizeof( FILE_ID_FULL_DIR_INFORMATION ) == sizeof( FILE_ALL_INFORMATION ), "different sizes!" ); fails on my end.

Looking at FILE_ALL_INFORMATION and FILE_ID_FULL_DIR_INFORMATION has me concerned ( but not certain due to alignment weirdness ) that this might not yield valid data. CreationTime for example, is the third field in FILE_ID_FULL_DIR_INFORMATION, but the first in FILE_ALL_INFORMATION - they're both file times, so I think it's plausible that if true, this might've been missed in testing.

I've tried building from scratch, and when I got to the b2 libs/afio/build step, I hit this:

C:\Users\Alexander Riccio\boost\boost_1_57_0>"C:\Users\Alexander Riccio\boostbuild_1_57\bin\b2.exe" libs/afio/build
warning: No toolsets are configured.
warning: Configuring default toolset "msvc".
warning: If the default is wrong, your build may not work correctly.
warning: Use the "toolset=xxxxx" option to override our guess.
warning: For more configuration options, please consult
warning: http://boost.org/boost-build2/doc/html/bbv2/advanced/configuration.html
Performing configuration checks

    - symlinks supported       : no  (cached)
    - junctions supported      : yes (cached)
    - hardlinks supported      : yes (cached)
...patience...
...patience...
...found 2560 targets...
...updating 4 targets...
compile-c-c++ bin.v2\libs\afio\build\msvc-12.0\debug\threading-multi\afio.obj
afio.cpp
c:\users\alexander riccio\boost\boost_1_57_0\boost\afio\config.hpp(58) : fatal error C1083: Cannot open include file: 'bindlib/include/import.h': No such file or directory

    call "C:\Users\ALEXAN~1\AppData\Local\Temp\b2_msvc_12.0_vcvarsall_x86.cmd" >nul
cl /Zm800 -nologo @"bin.v2\libs\afio\build\msvc-12.0\debug\threading-multi\afio.obj.rsp"

...failed compile-c-c++ bin.v2\libs\afio\build\msvc-12.0\debug\threading-multi\afio.obj...
...skipped <pbin.v2\libs\afio\build\msvc-12.0\debug\threading-multi>boost_afio-vc120-mt-gd-1_57.dll for lack of <pbin.v2\libs\afio\build\msvc-12.0\debug\threading-multi>afio.obj...
...skipped <pbin.v2\libs\afio\build\msvc-12.0\debug\threading-multi>boost_afio-vc120-mt-gd-1_57.lib for lack of <pbin.v2\libs\afio\build\msvc-12.0\debug\threading-multi>afio.obj...
...skipped <pbin.v2\libs\afio\build\msvc-12.0\debug\threading-multi>boost_afio-vc120-mt-gd-1_57.pdb for lack of <pbin.v2\libs\afio\build\msvc-12.0\debug\threading-multi>afio.obj...
...failed updating 1 target...
...skipped 3 targets...

specifically: c:\users\alexander riccio\boost\boost_1_57_0\boost\afio\config.hpp(58) : fatal error C1083: Cannot open include file: 'bindlib/include/import.h': No such file or directory

The documentation says in several places that AFIO is a "Boost.BindLib based Boost library", so I doubt this in intended. Perhaps that's just evidence of a mistake on my part?

I can't spot BindLib anywhere in Boost 1.57, or in AFIO itself, so I'm wondering if it is somehow specially configured on your machine?

Of course, the fact that you're the BindLib dev is maximally convenient ;)

ReFS doesn't support compressed files, hard links nor GUID paths, so these tests error out.

As stated in the title. The function is documented in the doxygen version of the documentation.

There is a lot of conflicting information out there regarding the need for directory fsyncing on Linux, and exactly when and when not one should fsync directories (why oh why can't Linux just be POSIX conformant! This isn't an issue on FreeBSD or any other platform).

Furthermore, directory fsyncing seems to be mostly a ext2/ext3/ext4 problem. BTRFS, XFS, JFS and others from what I know don't have this problem, but I really ought to confirm for sure.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3562.pdf
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3634.pdf

The less toy example has a bug in the parsing of the input files. It loops, incrementing argc instead of n, as it should.

Even after fixing that I cannot get the program to work, and I am unsure exactly what is wrong. The output file for the concatenated inputs is created and truncated to the correct size, but no data ever gets written to this file. If I use large files I have seen std::invalid_argument exceptions and even segfaults. I think the problem may be related to the buffers; I have tried using a simple std::vector<char> for the file_buffer_type, but this makes no difference.

UPDATE: So, I've been hacking a bit at this, and it turns out the buffer is not the problem. The problem seems to be here:

// Schedule a filling of buffer from offset o after last has completed
auto readchunk=dispatcher->read(make_async_data_op_req(last, buffer->data(),
    thischunk, o));
// Schedule a writing of buffer to offset offset+o after readchunk is ready
auto writechunk=dispatcher->write(make_async_data_op_req(readchunk,
    buffer->data(), thischunk, offset+o));

The precondition for the write is the previous readchunk future. This holds an io_op to the input file, so we are trying to write to the input file again. This file is opened for reading only, so the operation fails.

We're stuck until FreeBSD fixes their lack of __cxa_thread_atexit which causes thread local cleanup to not work.

gcc.compile.c++ bin.v2/libs/afio/build/gcc-4.7/debug/threading-multi/afio.o
/usr/include/c++/4.7/type_traits: In substitution of 'template<class _From1, class _To1> static decltype ((__test_aux<_To1>(declval<_From1>()), std::__sfinae_types::__one())) std::__is_convertible_helper<_From, _To, false>::__test(int) [with _From1 = _From1; _To1 = _To1; _From = const boost::afio::detail::async_file_io_dispatcher_op&; _To = boost::afio::detail::async_file_io_dispatcher_op] [with _From1 = const boost::afio::detail::async_file_io_dispatcher_op&; _To1 = boost::afio::detail::async_file_io_dispatcher_op]':
/usr/include/c++/4.7/type_traits:1258:70: required from 'constexpr const bool std::__is_convertible_helper<const boost::afio::detail::async_file_io_dispatcher_op&, boost::afio::detail::async_file_io_dispatcher_op, false>::value'
/usr/include/c++/4.7/type_traits:1263:12: required from 'struct std::is_convertible<const boost::afio::detail::async_file_io_dispatcher_op&, boost::afio::detail::async_file_io_dispatcher_op>'
/usr/include/c++/4.7/type_traits:116:12: required from 'struct std::_and<std::is_convertible<const unsigned int&, const unsigned int>, std::is_convertible<const boost::afio::detail::async_file_io_dispatcher_op&, boost::afio::detail::async_file_io_dispatcher_op> >'
/usr/include/c++/4.7/bits/stl_pair.h:113:38: required from 'struct std::__is_direct_constructible_impl<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, const std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>&>'
/usr/include/c++/4.7/type_traits:116:12: required from 'struct std::_and<std::is_destructible<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> >, std::__is_direct_constructible_impl<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, const std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>&> >'
/usr/include/c++/4.7/type_traits:778:12: [ skipping 5 instantiation contexts ]
/usr/include/c++/4.7/type_traits:929:12: required from 'struct std::__is_copy_constructible_impl<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, false>'
/usr/include/c++/4.7/type_traits:935:12: required from 'struct std::is_copy_constructible<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> >'
/usr/include/c++/4.7/bits/alloc_traits.h:542:12: required from 'struct std::__is_copy_insertable<std::allocator<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> > >'
/usr/include/c++/4.7/bits/alloc_traits.h:562:63: required from 'class std::__unordered_map<unsigned int, boost::afio::detail::async_file_io_dispatcher_op, std::hash, std::equal_to, std::allocator<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> >, false>'
/usr/include/c++/4.7/bits/unordered_map.h:264:11: required from 'class std::unordered_map<unsigned int, boost::afio::detail::async_file_io_dispatcher_op>'
libs/afio/src/afio.cpp:381:101: required from here
libs/afio/src/afio.cpp:371:3: error: 'boost::afio::detail::async_file_io_dispatcher_op::async_file_io_dispatcher_op(const boost::afio::detail::async_file_io_dispatcher_op&)' is private
In file included from libs/afio/src/../../../boost/afio/afio.hpp:19:0,
from libs/afio/src/afio.cpp:22:
/usr/include/c++/4.7/type_traits:1252:2: error: within this context
/usr/include/c++/4.7/type_traits: In substitution of 'template<class _From1, class _To1> static decltype ((__test_aux<_To1>(declval<_From1>()), std::__sfinae_types::__one())) std::__is_convertible_helper<_From, _To, false>::__test(int) [with _From1 = _From1; _To1 = _To1; _From = const boost::afio::detail::async_file_io_dispatcher_op&; _To = boost::afio::detail::async_file_io_dispatcher_op] [with _From1 = const boost::afio::detail::async_file_io_dispatcher_op&; _To1 = boost::afio::detail::async_file_io_dispatcher_op]':
/usr/include/c++/4.7/type_traits:1258:70: required from 'constexpr const bool std::__is_convertible_helper<const boost::afio::detail::async_file_io_dispatcher_op&, boost::afio::detail::async_file_io_dispatcher_op, false>::value'
/usr/include/c++/4.7/type_traits:1263:12: required from 'struct std::is_convertible<const boost::afio::detail::async_file_io_dispatcher_op&, boost::afio::detail::async_file_io_dispatcher_op>'
/usr/include/c++/4.7/type_traits:116:12: required from 'struct std::_and<std::is_convertible<const unsigned int&, const unsigned int>, std::is_convertible<const boost::afio::detail::async_file_io_dispatcher_op&, boost::afio::detail::async_file_io_dispatcher_op> >'
/usr/include/c++/4.7/bits/stl_pair.h:113:38: required from 'struct std::__is_direct_constructible_impl<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, const std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>&>'
/usr/include/c++/4.7/type_traits:116:12: required from 'struct std::_and<std::is_destructible<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> >, std::__is_direct_constructible_impl<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, const std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>&> >'
/usr/include/c++/4.7/type_traits:778:12: [ skipping 5 instantiation contexts ]
/usr/include/c++/4.7/type_traits:929:12: required from 'struct std::__is_copy_constructible_impl<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, false>'
/usr/include/c++/4.7/type_traits:935:12: required from 'struct std::is_copy_constructible<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> >'
/usr/include/c++/4.7/bits/alloc_traits.h:542:12: required from 'struct std::__is_copy_insertable<std::allocator<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> > >'
/usr/include/c++/4.7/bits/alloc_traits.h:562:63: required from 'class std::__unordered_map<unsigned int, boost::afio::detail::async_file_io_dispatcher_op, std::hash, std::equal_to, std::allocator<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> >, false>'
/usr/include/c++/4.7/bits/unordered_map.h:264:11: required from 'class std::unordered_map<unsigned int, boost::afio::detail::async_file_io_dispatcher_op>'
libs/afio/src/afio.cpp:381:101: required from here
libs/afio/src/afio.cpp:371:3: error: 'boost::afio::detail::async_file_io_dispatcher_op::async_file_io_dispatcher_op(const boost::afio::detail::async_file_io_dispatcher_op&)' is private
In file included from libs/afio/src/../../../boost/afio/afio.hpp:19:0,
from libs/afio/src/afio.cpp:22:
/usr/include/c++/4.7/type_traits:1252:2: error: within this context
/usr/include/c++/4.7/type_traits: In instantiation of 'constexpr const bool std::__is_convertible_helper<const boost::afio::detail::async_file_io_dispatcher_op&, boost::afio::detail::async_file_io_dispatcher_op, false>::value':
/usr/include/c++/4.7/type_traits:1263:12: required from 'struct std::is_convertible<const boost::afio::detail::async_file_io_dispatcher_op&, boost::afio::detail::async_file_io_dispatcher_op>'
/usr/include/c++/4.7/type_traits:116:12: required from 'struct std::_and<std::is_convertible<const unsigned int&, const unsigned int>, std::is_convertible<const boost::afio::detail::async_file_io_dispatcher_op&, boost::afio::detail::async_file_io_dispatcher_op> >'
/usr/include/c++/4.7/bits/stl_pair.h:113:38: required from 'struct std::__is_direct_constructible_impl<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, const std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>&>'
/usr/include/c++/4.7/type_traits:116:12: required from 'struct std::_and<std::is_destructible<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> >, std::__is_direct_constructible_impl<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, const std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>&> >'
/usr/include/c++/4.7/type_traits:778:12: required from 'struct std::__is_direct_constructible_new_safe<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, const std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>&>'
/usr/include/c++/4.7/type_traits:856:12: [ skipping 4 instantiation contexts ]
/usr/include/c++/4.7/type_traits:929:12: required from 'struct std::__is_copy_constructible_impl<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op>, false>'
/usr/include/c++/4.7/type_traits:935:12: required from 'struct std::is_copy_constructible<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> >'
/usr/include/c++/4.7/bits/alloc_traits.h:542:12: required from 'struct std::__is_copy_insertable<std::allocator<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> > >'
/usr/include/c++/4.7/bits/alloc_traits.h:562:63: required from 'class std::__unordered_map<unsigned int, boost::afio::detail::async_file_io_dispatcher_op, std::hash, std::equal_to, std::allocator<std::pair<const unsigned int, boost::afio::detail::async_file_io_dispatcher_op> >, false>'
/usr/include/c++/4.7/bits/unordered_map.h:264:11: required from 'class std::unordered_map<unsigned int, boost::afio::detail::async_file_io_dispatcher_op>'
libs/afio/src/afio.cpp:381:101: required from here
libs/afio/src/afio.cpp:371:3: error: 'boost::afio::detail::async_file_io_dispatcher_op::async_file_io_dispatcher_op(const boost::afio::detail::async_file_io_dispatcher_op&)' is private
In file included from libs/afio/src/../../../boost/afio/afio.hpp:19:0,
from libs/afio/src/afio.cpp:22:
/usr/include/c++/4.7/type_traits:1258:70: error: within this context

All of the annotations in nt_kernel_stuff are commented out....what's up with that?

Is it a MinGW thing? I think you can include nosal.h?

Immediately after filing #61 , I went ahead and made the change in my headers, and the test code I'm playing with failed to compile, due to an undeclared identifier for a format string, in afio_iocp.ipp at line 466.

Everything works fine when it is removed.

Usefully, this FreeBSD 10 setup is almost that for Apple Mac OS X, and therefore if it works here it almost certainly should cover OS X too.

/analyze found a read overrun in ErrorHandling.ipp.

The call to FormatMessageA at line 31 may fail ( as SAL would describe it ), and len equals zero.

The read at line 33 (to Remove annoying CRLF at end of message sometimes) then accesses buffer[len-1], which is buffer[0-1].