Re: [netcdfgroup] netCDF file close error on Windows Visual Studio 2015

I successfully ran the test program in two situations:
1. 32 bit cygwin
2. 64-bit centos7
I used the latest github master.
=Dennis Heimbigner
  Unidata


On 11/8/2018 11:09 AM, Mark Rivers wrote:
I realized that netCDF 4.1.3 that I was using was written in 2011, so it was not unreasonable to have problems with Visual Studio 2015, since it cannot have possibly been tested with it.
I have therefore updated my application to use the latest netCDF version, 4.6.1.
I found exactly the same problem.  I can write netCDF “classic” files over 2GB with no problems, but calling nc_close() fails if the file is larger than 2 GB. This happens with both Visual Studio 2015 and 2017.
A word about my setup.  My application is a file writing plugin for the areaDetector framework in the EPICS real-time control system.  I am using the EPICS build system, which allows a single Makefile to build for Linux, Windows, vxWorks, MacOS, and RTEMs operating systems.  I am using the netCDF source code without modification, but I am using a different Makefile.
On Linux I am running “configure” and using the resulting config.h files to build with.
On Windows I started with the Linux config.h file and manually edited it to work.
I finally figured out the problem.  The error was coming because ncio_filesize() was returning an error in NC3_close().  That is coming from this function in posixio.c 

static int

ncio_px_filesize(ncio *nciop, off_t *filesizep)

{

     /* There is a problem with fstat on Windows based systems

           which manifests (so far) when Config RELEASE is built.

           Use _filelengthi64 isntead. */

#ifdef HAVE_FILE_LENGTH_I64

     __int64 file_len = 0;

     if( (file_len = _filelengthi64(nciop->fd)) < 0) {

           return errno;

     }

     *filesizep = file_len;

#else

    struct stat sb;

    assert(nciop != NULL);

    if (fstat(nciop->fd, &sb) < 0)

     return errno;

    *filesizep = sb.st_size;

#endif

     return NC_NOERR;

}
The comments in the code led me to realize that perhaps I needed to define HAVE_FILE_LENGTH_I64 in my config.h file when using Visual Studio 2015 and 2017. This macro does not exist at all, defined or undefined, in the config.h generated with configure on Linux.  Once I defined that macro to 1 things worked fine on both Visual Studio 2015 and 2017.
It would be helpful if the netCDF distribution contained an example config.h file for Windows.  Can someone from Unidata provide me with such a file? 

I have a couple of other questions/problems.

I ran configure on 64-bit Linux as follows:
$ ./configure --disable-fortran --disable-netcdf-4 --disable-f90 --disable-dap --disable-cdf5 --disable-parallel4
So I tried to disable cdf5.  However, the resulting config.h file contains these lines: 

/* if true, enable CDF5 Support */

#define ENABLE_CDF5 1

/* if true, enable CDF5 Support */

#define USE_CDF5 1

Is this normal?  It looks like it is ignoring my –disable-cdf5 option?

When compiling netCDF 4.6.1 on Visual Studio 2010 I get an error:
cl               -nologo -FC -D__STDC__=0 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE   -Ox -Oy-   -W3     -DHAVE_CONFIG_H    -MT    -I. -I../O.Common -I. -I. -I../os/WIN32 -I../os/default -I.. -I../../../supportApp/netCDFSrc/inc -I../../../supportApp/netCDFSrc/libsrc -I../../../supportApp/netCDFSrc/libdispatch -I../../../supportApp/netCDFSrc/liblib -I../../../include/compiler/msvc -I../../../include/os/WIN32 -I../../../include -IJ:/epics/devel/asyn-4-33/include -IJ:/epics/devel/areaDetector-3-3-2/ADSupport/include/os/WIN32 -IJ:/epics/devel/areaDetector-3-3-2/ADSupport/include -IJ:/epics/devel/areaDetector-3-3-2/ADCore/include -IH:/epics-devel/base-7.0.1/include/compiler/msvc -IH:/epics-devel/base-7.0.1/include/os/WIN32 -IH:/epics-devel/base-7.0.1/include        -c ../../../supportApp/netCDFSrc/libdispatch/nchashmap.c 

nchashmap.c
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(157) : error C2275: 'NC_hentry' : illegal use of this type as an expression
       j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\inc\nchashmap.h(27) : see declaration of 'NC_hentry'
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(157) : error C2065: 'entry' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(158) : error C2065: 'entry' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(158) : error C2223: left of '->flags' must point to struct/union
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(160) : error C2065: 'entry' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(160) : error C2223: left of '->data' must point to struct/union
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(163) : error C2065: 'entry' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(163) : error C2223: left of '->flags' must point to struct/union
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(164) : error C2065: 'entry' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(164) : error C2223: left of '->data' must point to struct/union
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(165) : error C2065: 'entry' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(165) : error C2223: left of '->hashkey' must point to struct/union
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(166) : error C2065: 'entry' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(166) : error C2223: left of '->key' must point to struct/union
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(267) : error C2143: syntax error : missing ';' before 'type'
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(269) : error C2065: 'm' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(269) : error C2065: 'm' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(270) : error C2065: 'm' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(271) : error C2065: 'm' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(272) : error C2065: 'm' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(273) : error C2065: 'm' : undeclared identifier
j:\epics\devel\areadetector\adsupport\supportapp\netcdfsrc\libdispatch\nchashmap.c(274) : error C2065: 'm' : undeclared identifier
make[1]: *** [H:/epics-devel/base-7.0.1/configure/RULES_BUILD:232: nchashmap.obj] Error 2
make[1]: Leaving directory 'J:/epics/devel/areaDetector/ADSupport/supportApp/netCDFSrc/O.windows-x64-static-vs2010'
make: *** [H:/epics-devel/base-7.0.1/configure/RULES_ARCHS:58: install.windows-x64-static-vs2010] Error 2 

Any idea how to fix this?

Thanks,

Mark

*From:* Mark Rivers
*Sent:* Tuesday, November 6, 2018 9:09 PM
*To:* Mark Rivers <rivers@xxxxxxxxxxxxxxxxx>; 'netcdfgroup@xxxxxxxxxxxxxxxx' <netcdfgroup@xxxxxxxxxxxxxxxx> 
*Subject:* RE: netCDF file close error on Windows Visual Studio 2015
I was about to write a small test program to demonstrate this problem, when I realized I had already written one to demonstrate another problem back in 2011.  It is attached.  It writes 4100 1M arrays, and prints any error message when closing the file.
This is the end of the output when running the program on Linux (Centos 7 64-bit).  It works fine. 

writing record 4098/4100

writing record 4099/4100

writing record 4100/4100

Closing file test.nc

This on Windows with the VS 2010 compiler. It also works fine.

writing record 4098/4100

writing record 4099/4100

writing record 4100/4100

Closing file test.nc
This is on Windows with the VS 2015 compiler.  It returns an “Unknown error” when closing the file. 

writing record 4098/4100

writing record 4099/4100

writing record 4100/4100

Closing file test.nc

error=Unknown error

Thanks,

Mark
*From:* netcdf group-bounces@xxxxxxxxxxxxxxxx <mailto:group-bounces@xxxxxxxxxxxxxxxx> <netcdfgroup-bounces@xxxxxxxxxxxxxxxx <mailto:netcdfgroup-bounces@xxxxxxxxxxxxxxxx>> *On Behalf Of *Mark Rivers 
*Sent:* Tuesday, November 6, 2018 8:42 PM
*To:* 'netcdfgroup@xxxxxxxxxxxxxxxx' <netcdfgroup@xxxxxxxxxxxxxxxx <mailto:netcdfgroup@xxxxxxxxxxxxxxxx>> *Subject:* [netcdfgroup] netCDF file close error on Windows Visual Studio 2015 

Folks,
I am running a C application built with netCDF 4.1.3.  I am using 64-bit offset files, not HDF5.  I am creating files larger than 2 GB, typically 4GB or 8 GB.
It is working fine on Linux, and on Windows when building with Visual Studio 2010.
When building with Visual Studio 2015 it works fine with files less than 2GB.  However, when the file size exceeds 2GB I get an error closing the file.  The file is actually written fine, and I can process the data for an 8GB file.  However, because the file is not closed properly I cannot delete the file without closing the application. 

Is this a known problem, and if so is it fixed in a later release?

Thanks,

Mark



_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/