I get an IOError: Permission denied exception when trying to open a file, quickly after deleting it. I tested this code on Windows 8 64-bit, on both Python 2.6 and 2.7. It does not occur all the time, but occurred frequently enough to bother me.

My test code:

fn = 'asdf.txt'
count = -1

while True:
  count += 1
  log('Try:', count, '\n')
  f = openfile(fn, True)
  f.close()
  removefile(fn)

Output: (truncated some lines to make it take up less space)

PS C:\Users\leonw_000\Dropbox\Poly\Information Security\hw4\seattlelib> python2.6 repy.py restrictions.default test.repy
Try: 1
...
Try: 28
Try: 29
Internal Error

---
Uncaught exception!

---
Following is a full traceback, and a user traceback.
The user traceback excludes non-user modules. The most recent call is displayed last.

Full debugging traceback:
  "C:\Users\leonw_000\Dropbox\Poly\Information Security\hw4\seattlelib\namespace.py", line 1206, in wrapped_function
  "C:\Users\leonw_000\Dropbox\Poly\Information Security\hw4\seattlelib\emulfile.py", line 179, in emulated_open
  "C:\Users\leonw_000\Dropbox\Poly\Information Security\hw4\seattlelib\emulfile.py", line 293, in __init__

User traceback:

Exception (with type 'exceptions.IOError'): [13](Errno) Permission denied: 'C:\\Users\\leonw_000\\Dropbox\\Poly\\Information Security\\hw4\\seattlelib\\asdf.txt'

---
PS C:\Users\leonw_000\Dropbox\Poly\Information Security\hw4\seattlelib>

While I know this is a slight exaggeration, but I've run into this problem when writing code that repeatedly needs to regenerate a file that is used for testing purposes. I can easily change my code to use a randomly generated filename to avoid this problem, but I think we might do something to protect the user against this, as I couldn't catch this exception in my code.

In a little intercontinental debugging session, Lai Wang and I discovered a source of problems for RepyV2 on Android: RepyV2 was branched off before many of the Android fixes (r4853, r5634, #1084 and fixes, r4905, r4881, ) were introduced.

r4853 is direly missing. Without it we can't even get repy.py (V2) print its usage options and not crash.

Somehow, we need to add good support for the owner of the computer to select a set of security layers. These should be used by any sandboxed process that executes code on the machine.

The challenges include ensuring that the sandboxed program can't overwrite the code and persisting across vessel splits / joins. This mostly needs to be added for V2 (it's not as relevant to V1).

We need to deploy NAT forwarders on several nodes so nodes behind NAT can be accessed using the forwarders. The forwarders currently reside in seattle/branches/repy_v2/shims/nat_forwarder.

When running python2.6 utf.py -f ut_repyv2api_openfileperformsresourceaccounting.py, it prints:

'''We should have used 4096 more fileread! After: 7767.18577576 Before: 4096.0'''

nmclient.repy needs to be changed in the RepyV2 version in order to get it working with RepyV2 nodemanager. The current version of nmclient.repy only allows us to run RepyV1 code on vessels. Recently we added a new call in the nodemanager API (StartVesselEx) that allows us to run RepyV2 code as well as RepyV1 code on vessels. nmclient.repy needs to be modified so users are able to use it to upload and run RepyV2 code on vessels.

On Ubuntu 11.04, x86_64, Repy unit tests that pass on python2.6 fail with python2.7.1. This is true as of r4983.
Output is attached.

Add unit tests for RepyV2 UDP behavior.

On Ubuntu 11.04, x86_64, with Python 2.7.1+ the repyv2api unit tests all pass. They also pass with Python 2.7.2, compiled from source.

On Ubuntu 11.10, x86_64, with Python 2.7.2+, some of the unit tests fail with a message:

Exception exception_hierarchy.RunBuiltinException: RunBuiltinException() in <bound method Popen.__del__ of <subprocess.Popen object at 0x1dbe6d0>> ignored

The problem appears to be related to socket.close(), as commenting out s.close() in the following code allows it to pass the test properly:

#This unit test checks the sendmessage() API call.

#pragma repy
#pragma repy restrictions.twoports

s = listenformessage('127.0.0.1', 12345)

data = "HI"*8

sendmessage('127.0.0.1', 12345, data, '127.0.0.1', 12346)

(rip, rport, mess) = s.getmessage()

s.close()

if mess != data:
  log("Mismatch!",'\n')

Repyv2 is not compatible with windows mobile.
Therefore, any windows mobile-related code should be removed from repyv2.

Running the attached security layer together with the sample code runs with no issues on Linux, with Python 2.6.6, but times out on Windows, even with a moderately powerful CPU.

The function that is being 'interfaced' is sleep(seconds), with a simple security layer

This issue was reported on the following configurations:
Laptops:
Win 7 (x86), python 2.6.4, 2.1 Ghz Pentium Dual Core
Win 7 (x86), python 2.5, 1.6 Ghz Atom

This was last tested on revision ~4861 or so.

We need to write a library that discovers whether a node is behind a NAT. Ideally users should be able to import this library and call a function that returns a boolean value.

There are still portions of repyv1 code that are included in the repyv2 branch that are incompatible with repyv2. These should be removed.

The getresources() call is not correctly mapped into the namespace. All calls (except the Lind specific ones) should be mapped in.

The library file sockettimeout.repy needs to be ported to Repy V2.

These lines of the unit tests are giving us problems:

# Open random files
for x in xrange(MAX_FILES):
  filehandles[x] = openfile(files[x], False)

openfile does not allow uppercase letters in filenames. However, README.txt is the first file that is obtained. For this below sample, I commented out the openfile call, and logged the filenames instead:

README.txt
testsemaphore_simple.py
loggingrepy.pyc
onehopdetourshim.repy
nanny.py

Perhaps all we need to do is to modify the unit test to skip any files with capitals.

If the ip address of the local machine changes while doing a getmyip() call then it blocks forever. We need it to either return the new updated ip address or raise an error. This is required for the mobility shim.

The call Thread.is_alive() is new in Python 2.6 it seems, and we should not use it. Instead we should be using either Thread.isAlive() or find an alternate solution. There are many reference to Thread.is_alive() in the RepyV2 branch in our repository, and these should be cleaned up.

The all-logsec security layer fails because it differs from some of the calls. Namely: gethostbyname needs an argument list and file.writeat returns None.

In repy, creating a variable adds an entry in the _context dictionary. This allows them to be accessed similarly to global variables, through `_context[Repyportability does not expose these calls.

For example,

foo = 23
def printmyfoo():
  log(_context["foo"]("varname"]`.))
printmyfoo()

runs without any problems, and correctly prints 23 - when called with repy ( python repy.py restrictions.default test_case.repy)

However, when called with repyportability, it throws a keyerror:

Code:

import repyhelper
repyhelper.translate_and_import("test_case.repy")

Error:

  File "<string>", line 1, in <module>
  File "repyhelper.py", line 439, in translate_and_import
    _import_file_contents_to_caller_namespace(modulename, preserve_globals)
  File "repyhelper.py", line 469, in _import_file_contents_to_caller_namespace
    import_module = __import__(modulename)
  File "testCase.py", line 22, in <module>
    printmyfoo()
  File "testCase.py", line 20, in printmyfoo
    log(_context["foo"])
  File "safe.py", line 397, in __getitem__
    return self.__under__.__getitem__(key)
KeyError: 'foo'

All the nodemanager unit tests need to be ported over to Repy V2 and have to get them working before we can deploy Repy V2 nodemanager.

The library file sockettimeout.repy needs to be updated to use the Repy V2 network API. It is currently using the Repy V1 API calls.

Reported by Danny:

"""
I observed one situation in which the semantics of repy was broken. I
had a repy server listening bound to IP x. After it was started, I
changed the IP address to y. An openconnection on y would raise a
connection-refused error. On the other hand, openconnection on x gave

this error:

Uncaught exception!

Following is a full traceback, and a user traceback.
The user traceback excludes non-user modules. The most recent call is
displayed last.
Full debugging traceback:
"/tmp/seattle/namespace.py", line 1206, in wrapped_function
"/tmp/seattle/emulcomm.py", line 1373, in openconnection
"/tmp/seattle/emulcomm.py", line 1190, in _timed_conn_initialize
"", line 1, in connect
User traceback:
"", line 1, in connect
Exception (with class 'socket.error'): 113 No route to host

This problem concerns me. The mobility advertises the new IP address
whenever it changes, but there is a short gap between the detection of
this change and the successfully announcement of the new IP. If a
client connects during this gap, it will crash.
For now, I can avoid the problem by assuming this won't happen in the
evaluation.
"""

Sending the empty string through a repy v2 socket results in the following error:

Internal Error

---
Uncaught exception!

---
Following is a full traceback, and a user traceback.
The user traceback excludes non-user modules. The most recent call is displayed last.

Full debugging traceback:
  "/home/sportzer/seattle/temp/namespace.py", line 1208, in wrapped_function
  "/home/sportzer/seattle/temp/namespace.py", line 1132, in _process_retval

User traceback:

Exception (with class 'exception_hierarchy.InternalRepyError'): Function 'send' returned with unallowed return type <type 'int'> : Invalid retval type: Min value is 1.

---
Terminated

Basically, it looks like this is a special case where socket.send returns 0, but namespace enforces the restriction that the return value must be positive.

I don't think there is any point in keeping the --simple flag for repy V2. Now that we've dispensed with all of the multiple code calling nonsense, this is obsolete.

Suppose you are listening on a given local IP/port combination for TCP connections and get a socket for some remote IP/port using TCPServerSocket.getconnection(). The actual python socket is stored in a dictionary that uses the (protocol, local IP, local port, remote IP, remote port) 5-tuple as a key, and the repy socket stores this 5-tuple to uniquely identify it's connection. Now suppose that the connection is closed remotely and the same remote IP/port as before reconnects to the local IP/port. Calling TCPServerSocket.getconnection() again will return a new socket bound to the new connection, but this new socket will have the same 5-tuple as the first socket. Because the first socket was closed remotely, we don't necessarily know it was closed, so it will appear that we have two different connections with the same 5-tuple. What will happen in this case is getconnection won't check if the socket dictionary already has a mapping for the 5-tuple, and so it will overwrite the existing entry. As a result, both the first socket and the new one will use the new connection since they access their connections using the same 5-tuple.

This behavior is especially problematic if the first socket is closed either by invoking socket.close() or through garbage collection since this will result in the connection of the new socket being closed. Subsequent calls to send or recv by the new socket will result in !SocketClosedLocal being raised unexpectedly.

The attached test case triggers this behavior. (In Linux, Windows just raises !CleanupInProgressError)

There are several seattle library files, that have not yet been converted to repyV2 needs to be converted and added to the repy_v2 branch.

We need a method for balancing the load on the NAT forwarders. We don't want all the nodes behind a NAT to be using the same forwarder. Perhaps we can find a forwarder depending on the geographical location?

The placeholder textops.py in repy_v2/repy/textops.py can supersede the textops.py library in repy_v2/seattlelib/textops.py when you copy them all to the same directory. This causes textops calls to break.

(I saw this after running preparetest.py, I'm not sure why this happened since it's supposed to copy seattlelib ''after'' repy)

In Repy V2, adding items like safe._BUILTIN_OK.append("foo") does not work as expected. The reason is that safe_check also needs to be modified.

As a result, there is no good way to do something like add a command line option to repy that turns on / off the safety of some strings. This should really work with the dynamic settings that repy has in its context. I can think of hacked up ways to do this, but would prefer a suggestion that is clean and readable.

In repy v2, a new UDP socket is created for each call to sendmessage. Based on the results of testing, this is significantly less efficient than reusing an existing UDP socket, and is more likely to result in a abnormally large delay. Graphs are attached comparing the existing implementation to python socket behavior for creating a new socket each time and for reusing the same socket each time.

It would be nice if sendmessage was moved to be a method of the UDP socket class.

If you run Repy in Python 2.7, the "as" keyword in except clauses is allowed. This keyword does not work in Python 2.5 or 2.6.

The change in behavior causes WORA headaches. We should address this issue (probably by blocking the keyword) to avoid confusion.

(Reported by Jay Koven.)

The translation of ntp_time from repy v1 to repy v2 for the repy v2 node manager branch has a few issues. First, localip is used instead of ip on line 82. Secondly, it should raise an error on failure instead of returning False. Finally, the last two lines, which register the time method, should not be indented.

Additionally, sendmessage should take take an IP for it's first argument rather than a url, but looking at the repy v1 version of ntp_time I'm confused as to why it doesn't have the same problem (assuming that version actually works).

Per this stackoverflow thread,
"A shutdown() affects all copies of the socket while close() affects only the file descriptor in one process." Justin substantiates that when close()ing a socket in one thread, other threads blocked on the same socket will hang indefinitely. To work around that issue, socket I/O in RepyV2 was implemented in a non-blocking fashion, which increases the amount of CPU burned polling the socket for data (among other side effects).

We should evaluate the behavior of shutdown() vs close() on Linux in a simple multi-threaded Python program that, e.g., connects to a local instance of netcat. Following that, it will be interesting to learn whether the problem exists in comparable form on the other OSses.

All the nodemanager tests need to be updated and use the new nmclient.repy. This will have to be done after #1133 has been resolved.

Our sockets for RepyV2 hang. Below might be a fix to this issue.

As another example, when the call quality of a Skype session degrades, Skype attempts
to close its socket from a separate thread, as we discovered in Skype diagnosis
(Section~\ref{subsec-big_apps}). This is a known bug that
when a TCP or UDP socket is blocking on call \texttt{recv/recvfrom}, a
\texttt{close} call made on this socket from a different thread will keep the socket
blocking indefinitely. This bug has also been repeatedly reported
on Ruby and GCC bug trackers. \yanyan{cite TR.} \texttt{block\_tcp/udp\_close\_socket} in
Table~\ref{tab:semantic-unit-tests} are for this test case, where \sysname detected the
issues in both TCP and UDP correctly. Further, when \texttt{shutdown} is
called on the blocking socket, \texttt{recv/recvfrom} gets unblocked immediately by
raising an exception. This can be a suggested solution to the Skype call quality problem.

Some of the network tests fail when run with the rest of the module. When run individually, they pass for me.

justinc@nara:~/test2> python utf.py -m repyv2api
Testing module: repyv2api
    Running: ut_repyv2api_connectionsendwillnotblock.py         [ PASS ]
    Running: ut_repyv2api_fileclosereleasesresource.py          [ PASS ]
    Running: ut_repyv2api_openconnectionresourceforbidden.py    [ PASS ]
    Running: ut_repyv2api_listenforconn-resources.py            [ PASS ]
    Running: ut_repyv2api_sendmessage5ktest.py                  [ PASS ]
    Running: ut_repyv2api_listenforconn-badport.py              [ PASS ]
    Running: ut_repyv2api_filereadatbasictest.py                [ PASS ]
    Running: ut_repyv2api_openconnectionwithbadlocalip.py       [ PASS ]
    Running: ut_repyv2api_listenforconn-addrbinding.py          [ PASS ]
    Running: ut_repyv2api_openconnectiontimeout.py              [ PASS ]
    Running: ut_repyv2api_listencloselisten2.py                 [ FAIL ]
--------------------------------------------------------------------------------
Standard error :
..............................Produced..............................

---
Uncaught exception!

---
Following is a full traceback, and a user traceback.
The user traceback excludes non-user modules. The most recent call is displayed last.

Full debugging traceback:
  "repy.py", line 190, in main
  "/home/justinc/test2/virtual_namespace.py", line 113, in evaluate
  "/home/justinc/test2/safe.py", line 508, in safe_run
  "ut_repyv2api_listencloselisten2.py", line 17, in <module>
  "/home/justinc/test2/namespace.py", line 1206, in wrapped_function
  "/home/justinc/test2/emulcomm.py", line 1304, in openconnection
  "/home/justinc/test2/emulcomm.py", line 1170, in _timed_conn_initialize

User traceback:
  "ut_repyv2api_listencloselisten2.py", line 17, in <module>

Exception (with class 'exception_hierarchy.TimeoutError'): Timed-out connecting to the remote host!

---

..............................Expected..............................
None
--------------------------------------------------------------------------------
    Running: ut_repyv2api_getthreadnamebasictest.py             [ PASS ]
    Running: ut_repyv2api_createlockblocks.py                   [ PASS ]
    Running: ut_repyv2api_filewriteatperformsresourceaccounting.py [ PASS ]
    Running: ut_repyv2api_exitallstopsanotherthread.py          [ PASS ]
    Running: ut_repyv2api_filereadatperformsresourceaccounting.py [ PASS ]
    Running: ut_repyv2api_openfileconsumesfilehandles.py        [ PASS ]
    Running: ut_repyv2api_listenclosesend.py                    [ PASS ]
    Running: ut_repyv2api_fileclosemakesothercallsfail.py       [ PASS ]
    Running: ut_repyv2api_filecloseconcurrecy.py                [ PASS ]
    Running: ut_repyv2api_filewriteatsanitychecksargs.py        [ PASS ]
    Running: ut_repyv2api_virtualnamespace-eval.py              [ PASS ]
    Running: ut_repyv2api_createthreadissane.py                 [ PASS ]
    Running: ut_repyv2api_openfileperformsresourceaccounting.py [ PASS ]
    Running: ut_repyv2api_listenformessage-reopening.py         [ PASS ]
    Running: ut_repyv2api_connectionrecvwouldblock.py           [ PASS ]
    Running: ut_repyv2api_openfileduplicatefilefails.py         [ PASS ]
    Running: ut_repyv2api_listenformessage-exceptions.py        [ PASS ]
    Running: ut_repyv2api_filereadatpastendoffile.py            [ PASS ]
    Running: ut_repyv2api_filereadatsanitychecksargs.py         [ PASS ]
    Running: ut_repyv2api_listenforconn-cleanup.py              [ PASS ]
    Running: ut_repyv2api_createlocknonblocking.py              [ PASS ]
    Running: ut_repyv2api_virtualnamespace-init.py              [ PASS ]
    Running: ut_repyv2api_sleepbasictest.py                     [ PASS ]
    Running: ut_repyv2api_listfilesbasictest.py                 [ PASS ]
    Running: ut_repyv2api_listenforconn-args.py                 [ PASS ]
    Running: ut_repyv2api_createthreadhasresourcecontrols.py    [ PASS ]
    Running: ut_repyv2api_filewriteatbasictest.py               [ PASS ]
    Running: ut_repyv2api_connectionrecvremoteclose.py          [ PASS ]
    Running: ut_repyv2api_connectionsendremoteclose.py          [ PASS ]
    Running: ut_repyv2api_connectionduplicateclose.py           [ PASS ]
    Running: ut_repyv2api_openconnectionsanitychecksargs.py     [ PASS ]
    Running: ut_repyv2api_openconnectionfailswithoutlisten.py   [ PASS ]
    Running: ut_repyv2api_listenformessage-distinctfromtcp.py   [ PASS ]
    Running: ut_repyv2api_openfilesanitychecksargs.py           [ PASS ]
    Running: ut_repyv2api_unicodedisallowed.py                  [ PASS ]
    Running: ut_repyv2api_getlasterrorworks.py                  [ PASS ]
    Running: ut_repyv2api_virtualnamespacecontextsafety.py      [ PASS ]
    Running: ut_repyv2api_sendmessagecheckstypes.py             [ PASS ]
    Running: ut_repyv2api_randombytesissane.py                  [ PASS ]
    Running: ut_repyv2api_initialusevaluesaresane.py            [ PASS ]
    Running: ut_repyv2api_sendmessage-exceptions.py             [ PASS ]
    Running: ut_repyv2api_getresourcedataisvalid.py             [ PASS ]
    Running: ut_repyv2api_openconnectionduplicatetuple.py       [ PASS ]
    Running: ut_repyv2api_connectionserversendblocks.py         [ FAIL ]
--------------------------------------------------------------------------------
Standard error :
..............................Produced..............................

---
Uncaught exception!

---
Following is a full traceback, and a user traceback.
The user traceback excludes non-user modules. The most recent call is displayed last.

Full debugging traceback:
  "repy.py", line 190, in main
  "/home/justinc/test2/virtual_namespace.py", line 113, in evaluate
  "/home/justinc/test2/safe.py", line 508, in safe_run
  "ut_repyv2api_connectionserversendblocks.py", line 15, in <module>
  "/home/justinc/test2/namespace.py", line 1206, in wrapped_function
  "/home/justinc/test2/emulcomm.py", line 1304, in openconnection
  "/home/justinc/test2/emulcomm.py", line 1170, in _timed_conn_initialize

User traceback:
  "ut_repyv2api_connectionserversendblocks.py", line 15, in <module>

Exception (with class 'exception_hierarchy.TimeoutError'): Timed-out connecting to the remote host!

---

..............................Expected..............................
None
--------------------------------------------------------------------------------
    Running: ut_repyv2api_listfilesperformsresourceaccounting.py [ PASS ]
    Running: ut_repyv2api_listencloselisten.py                  [ PASS ]
    Running: ut_repyv2api_tcpserver_wouldblock.py               [ PASS ]
    Running: ut_repyv2api_exitallstopscurrentthread.py          [ PASS ]
    Running: ut_repyv2api_removefileperformsresourceaccounting.py [ PASS ]
    Running: ut_repyv2api_multipleopenconnections2.py           [ PASS ]
    Running: ut_repyv2api_listenforconn-dup.py                  [ PASS ]
    Running: ut_repyv2api_connectionsendwilleventuallyblock.py  [ PASS ]
    Running: ut_repyv2api_nannyupdatesresourceconsumption.py    [ PASS ]
    Running: ut_repyv2api_connectionrecvlocalclose.py           [ PASS ]
    Running: ut_repyv2api_connectionsendlocalclose.py           [ PASS ]
    Running: ut_repyv2api_randombytesisresourcelimited.py       [ PASS ]
    Running: ut_repyv2api_stoptimesaresane.py                   [ PASS ]
    Running: ut_repyv2api_openconnectionalreadyalistening.py    [ PASS ]
    Running: ut_repyv2api_sendmessagebasictest.py               [ PASS ]
    Running: ut_repyv2api_removefileissane.py                   [ PASS ]
    Running: ut_repyv2api_multipleopenconnections.py            [ PASS ]

Without changing my IP I get:

Following is a full traceback, and a user traceback.
The user traceback excludes non-user modules. The most recent call is displayed last.

Full debugging traceback:
"librepysocket", line 395, in accept_waiting
"/Users/adadgar/Projects/seattle/branches/repy_v2/test/namespace.py", line 909, in __do_func_call
"/Users/adadgar/Projects/seattle/branches/repy_v2/test/namespace.py", line 1161, in wrapped_function
"/Users/adadgar/Projects/seattle/branches/repy_v2/test/emulcomm.py", line 1981, in getmessage

User traceback:
"librepysocket", line 395, in accept_waiting

Exception (with class 'exception_hierarchy.LocalIPChanged'): The local ip 192.168.0.100 is no longer present on a system interface.

This became an issue for me when my ubuntu's update-manager refused to update because I had switched my python to python2.6.
Given the simplicity and the benefit, I think it is worthwhile to implement this fix.

Inability to have python installed as anything besides 'python', and it must be in the path

In Repy, the code that spawns a subprocess calls 'python' directly, instead of using the portable sys.executable.
By replacing these 4 instances of 'python' with sys.executable, users will not be forced to:

1. Add python to their path variable
2. Have python 2.5/2.6 as their default version of python

Note that sys will need to be imported in each of these files that does not do that already.

Safe.py

Line 276: Calls 'python' directly, without the full path. This means that you're forced to have this version of python in the path. This should be replaced with the executable python that is being used to launch repy.

Replace:
proc = subprocess.Popen(path_to_safe_check,stdin=subprocess.PIPE, stdout=subprocess.PIPE)

with:

proc = subprocess.Popen(path_to_safe_check,stdin=subprocess.PIPE, stdout=subprocess.PIPE)

sys.executable is the location of the python executable used to call this program. It exists for both Python 2.5 and 2.6.
Using this would allow users to not be required to make python 2.5/2.6 their default 'python' program.

utf.py

Line 301, same issue as above
Line 477, same issue as above

utfutil.py

Line 57, same issue as above

Creating a fresh copy of the repy v2 node manager from the repy_v2_nodemanager branch, API calls fail due to the time appearing to not be set.

  File "/home/sportzer/Documents/seattle/nm_v2_temp/nmrequesthandler.py", line 101, in handle_request
  File "/home/sportzer/Documents/seattle/nm_v2_temp/nmrequesthandler.py", line 227, in process_API_call
  File "/home/sportzer/Documents/seattle/nm_v2_temp/nmrequesthandler.py", line 257, in ensure_is_correctly_signed
  File "/home/sportzer/Documents/seattle/nm_v2_temp/fastsigneddata.py", line 285, in signeddata_iscurrent
  File "time_interface", line 162, in time_gettime
TimeError: Error from time_gettime(): time has not been set yet!

As far as I can tell, this is due to repyportability reinstantiating modules every time they are imported, or something equally weird.

We need to test librepysocket thoroughly and ensure that all the functions work properly. We need to ensure that the semantics for the API calls match properly since many of the seattle library files have not been ported over to Repy V2 yet.

It's unclear why getmyip() should raise an error if the host is not connected to the Internet. Yes, this is okay, according to the specification of the API, but really doesn't make much sense. It should choose a non-loopback IP instead (if available) or loopback if not.

This happens on my Linux laptop, but does not seem to happen on Prof Cappos's Mac laptop (possibly).

The error seems to come from the line:
sockobj.connect((external_ip, external_port))
in _get_localIP_to_remoteIP.

The error raised by the line looks like the following:
error 101 Network is unreachable

This is also being tracked on the AFFIX tracker as: https://affix.poly.edu/projects/project/ticket/6

Sebastian reports:

sendmessage raises a SocketWouldBlockError when it is called using a port that has an active udpserversocket already, so in order to achieve the desired functionality I have to close the listening socket on port 53 before I can send a reply. When I do this, somehow the number of outsockets increases by one, however future transactions do not seem to be impeded. As such, I suspect this is an erroneous iteration of the outsockets count, but I'm still looking in to that.

Some experiments need to be run in order to figure out what the ideal number of servers and clients each NAT forwarder should handle. Currently it defaults to 5 servers and 5 clients per server.

When you have a simple server/client in repyV2 and you close the connection remotely. You are still able to send one message across without getting an error. It isn't until the second send() when you get the SocketClosedRemote error.

The test file ut_repyv2api_connectionsendremoteclose.py is a good example of where it fails.

The default resource file has a random resource of 1000. However the currently implemented randombytes() function in repy V2 consumes 1024 random resources. This will cause randombytes to take while to return a value since it will wait until the resource consumed drains in order to ensure we don't consume more then 1000 random resource.

It is typical for most of the networking unit tests to fail when run on a Mac. (Example failure output below.) This is due to !CleanupInProgressErrors. We should handle this in an intelligent way. I'm going to investigate adding a short sleep / retry to appropriate tests and if this does not work then maybe the tests should choose unique ports in some range.

    Running: ut_repyv2api_connectionserversendblocks.py         [ FAIL ]
--------------------------------------------------------------------------------
Standard error :
..............................Produced..............................

---
Uncaught exception!

---
Following is a full traceback, and a user traceback.
The user traceback excludes non-user modules. The most recent call is displayed last.

Full debugging traceback:
  "repy.py", line 190, in main
  "/Users/justincappos/test2/virtual_namespace.py", line 113, in evaluate
  "/Users/justincappos/test2/safe.py", line 508, in safe_run
  "ut_repyv2api_connectionserversendblocks.py", line 15, in <module>
  "/Users/justincappos/test2/namespace.py", line 1206, in wrapped_function
  "/Users/justincappos/test2/emulcomm.py", line 1316, in openconnection
  "/Users/justincappos/test2/emulcomm.py", line 1104, in _conn_cleanup_check

User traceback:
  "ut_repyv2api_connectionserversendblocks.py", line 15, in <module>

Exception (with class 'exception_hierarchy.CleanupInProgressError'): The socket is being cleaned up by the operating system!

---

..............................Expected..............................
None
-------------------------------------------------------------------------------

The AST built by the compiler module is quite large. We really don't need it because our focus is on information we can derive without examining other tree nodes. Can we avoid this memory overhead?

Dennis Schwerdel reports that it's hard to call repy code externally. Specifically, for his integration with !ToMaTo ( http://tomato.german-lab.de/project ), he needs to be able to insert some new calls into the namespace and instantiate the repy code.

One hurdle to this is that we do too much in repy.py's main, especially involving the use of globals. We should make it possible to import main and do sane things with it.

In v2, Emulcomm is incapable of using sendmessage on a port which it is currently listening on for UDP datagrams. This should not be the case, because the underlying python API allows for such functionality.