[IPython-user] Fwd: Preparing for the 0.7.4 release

[Jörgen Stenarson]

DaveS skrev:
> On Thu, Mar 22 2007, Fernando Perez wrote:
> 
>> [ This was meant for the list...]
>>
>> ---------- Forwarded message ----------
>> From: Fernando Perez <fperez.net@gmail.com>
>> Date: Mar 22, 2007 7:01 PM
>> Subject: Re: [IPython-user] Preparing for the 0.7.4 release
>> To: Stefan van der Walt <stefan@sun.ac.za>
>>
>>
>> On 3/22/07, Stefan van der Walt <stefan@sun.ac.za> wrote:
>>> On Thu, Mar 22, 2007 at 05:28:56PM -0600, Fernando Perez wrote:
>>>> But there's one in particular that I'm a bit stumped on, the unicode bug:
>>>>
>>>> http://projects.scipy.org/ipython/ipython/ticket/129
>>>>
>>>> If anyone knows what to do here, I'd be very grateful for help
>>>> (patches, ideas, a reference on what to do, whatever).
>>> Hi Fernando
>>>
>>> The attached patch fixes the problem.  I didn't check to see whether
>>> it crops up anywhere else though, but I believe that is the right way
>>> to handle input from the terminal.
>>>
>>> Cheers
>>> Stéfan
>>>
>>> Index: IPython/iplib.py
>>> ===================================================================
>>> --- IPython/iplib.py    (revision 2164)
>>> +++ IPython/iplib.py    (working copy)
>>> @@ -1977,7 +1977,7 @@
>>>          """
>>>
>>>          try:
>>> -            line = raw_input_original(prompt)
>>> +            line = raw_input_original(prompt).decode(sys.stdin.encoding)
>>>          except ValueError:
>>>              warn("\n********\nYou or a %run:ed script called sys.stdin.close() or sys.stdout.close()!\nExiting IPython!")
>>>              self.exit_now = True
>>>
>> Thanks!!!
>>
>> http://projects.scipy.org/ipython/ipython/changeset/2168
>>
>> It works in all the tests I did quickly.  If anyone finds a condition
>> where this doesn't help, or knows a good reason why it might be the
>> wrong approach, please let me know or reopen the ticket.
>>
>> Otherwise, lots of brownie points to Stefan for this one.
>>
>> Cheers,
>>
>> f
> 
> Sorry, but I found a problem with this under emacs.  (WinXP NTEmacs 21.3)
> 
> In the cmd window, sys.stdin.encoding = 'cp850'.  In emacs
> sys.stdin.encoding = None.  This breaks ipython under emacs:

I have found more problems. Below is a transcript of a session in 
regular python with a cp850 terminal and ipython in the same terminal.
As you can see there was a bug introduced recently that gives an 
exception in the prefilter code if the line starts with a string 
containing non-ascii characters, this worked a couple of days ago.

The new raw_input patch seems to translate åäö to some kind of two-byte 
unicode when assigning to regular strings but it seems to work fine when 
assigning to unicode strings. I have been playing with 
eval('"åäö"'.decode("xxx")) (where xxx is a valid encoding) and 
eval('u"åäö"'.decode("xxx")) but I have not been able to find anything 
that works in both cases. I wonder if something other than eval is used 
for the regular python prompt.

/Jörgen


c:\python\packages>python
Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
pyreadline: 1.4.svn
 >>> "åäö"
'\x86\x84\x94'
 >>> u"åäö"
u'\xe5\xe4\xf6'
 >>>



c:\python\packages>ipython
Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)]
Type "copyright", "credits" or "license" for more information.

IPython 0.7.4.svn.r2159 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: "åäö"
---------------------------------------------------------------------------
exceptions.UnicodeEncodeError                        Traceback (most 
recent call
  last)

c:\python\ipython\IPython\iplib.py in 
multiline_prefilter(self=<IPython.iplib.In
teractiveShell object>, line=u'"\xe5\xe4\xf6"', continue_prompt=0)
    2243 out = []
    2244 for l in line.rstrip('\n').split('\n'):
-> 2245 out.append(self._prefilter(l, continue_prompt))
         out.append = <built-in method append of list object at 0x012FE490>
         self._prefilter = <bound method InteractiveShell._prefilter of 
<IPython.
iplib.InteractiveShell object at 0x00A22BB0>>
         l = u'"\xe5\xe4\xf6"'
         continue_prompt = 0
    2246 return '\n'.join(out)
    2247

c:\python\ipython\IPython\iplib.py in 
_prefilter(self=<IPython.iplib.Interactive
Shell object>, line=u'"\xe5\xe4\xf6"', continue_prompt=0)
    2147 # Let's try to find if the input line is a magic fn
    2148 oinfo = None
-> 2149 if hasattr(self,'magic_'+iFun):
         global hasattr = undefined
         self = <IPython.iplib.InteractiveShell object at 0x00A22BB0>
         iFun = u'"\xe5\xe4\xf6"'
    2150 # WARNING: _ofind uses getattr(), so it can consume generators and
    2151 # cause other side effects.

UnicodeEncodeError: 'ascii' codec can't encode characters in position 
7-9: ordin
al not in range(128)

In [2]: a="åäö"

In [3]: a

Out[3]: '\xc3\xa5\xc3\xa4\xc3\xb6'

In [4]: a=u"åäö"

In [5]: a

Out[5]: u'\xe5\xe4\xf6'