Dreaming of new generation IDE

Hello,

I am sitting here for quite some time, but usually keep silent ;-) I
use Python since 2003 both "professionally" and for my hobby projects
and love it a much.
I notice however, that "maintaining" existing/older python code is may
be not so enjoyable task. It may be even harder than supporting old
code written in some type of "static" languages (like Java or C++).
Surely "dynamic" nature of python comes with price.

Finally I develop a feeling that strong instrumentation / tools can
bring us the best of two worlds. That I am dreaming on is an absolute
new type/class of IDE suitable for Python and potentially for other
dynamic-type languages. Instead of current text-oriented IDEs, it
should be a database-centric and resemble current CAD systems instead
of being just "fancy text editor". Source text should be an output
product of that CAD and not a "source material" itself.

Well. I understand that it is a very ambitious and experimental stuff.
Great efforts and motivation needed even to get something "runnable".
So I am looking for someone to get in kind of "virtual partnership".
If someone interesting it that soft of stuff, I would like to talk and
discuss this system.

Vladimir Ignatov

Vladimir Ignatov <kmisoft@gmail.com> writes:
> I notice however, that "maintaining" existing/older python code is may
> be not so enjoyable task. It may be even harder than supporting old
> code written in some type of "static" languages (like Java or C++).
> Surely "dynamic" nature of python comes with price.

Yes, this is a well known drawback of dynamic typing. The usual remedy
is lots and lots of test automation (a full suite of unit and
integration tests that you run on every build, that check the properties
of basically every function in your program).

> Instead of current text-oriented IDEs, it
> should be a database-centric and resemble current CAD systems

I've never used a current CAD system, so I can't make any sense of this.
I don't see how databases would help.

Paul Rubin, 03.02.2010 14:07:
>> Instead of current text-oriented IDEs, it
>> should be a database-centric and resemble current CAD systems
>
> I've never used a current CAD system, so I can't make any sense of this.
> I don't see how databases would help.

Just like they help in current IDEs to index the source code.

Stefan

On Feb 3, 12:10*pm, Vladimir Ignatov <kmis...@gmail.com> wrote:
> Hello,
>
> I am sitting here for quite some time, but usually keep silent ;-) I
> use Python since 2003 both "professionally" and for my hobby projects
> and love it a much.
> I notice however, that "maintaining" existing/older python code is may
> be not so enjoyable task. It may be even harder than supporting old
> code written in some type of "static" languages (like Java or C++).
> Surely "dynamic" nature of python comes with price.
>
> Finally I develop a feeling that strong instrumentation / tools can
> bring us the best of two worlds. That I am dreaming on is an absolute
> new type/class of IDE suitable for Python and potentially for other
> dynamic-type languages. Instead of current text-oriented IDEs, it
> should be a database-centric and resemble current CAD systems instead
> of being just "fancy text editor". Source text should be an output
> product of that CAD and not a "source material" itself.
>
> Well. I understand that it is a very ambitious and experimental stuff.
> Great efforts and motivation needed even to get something "runnable".
> So I am looking for someone to get in kind of "virtual partnership".
> If someone interesting it that soft of stuff, I would like to talk and
> discuss this system.
>
> Vladimir Ignatov

The maintenance thing may be true but for me that doesn't outweigh the
clear benefits I get from using Python i.s.o. e.g. C++: the fact that
I have much less code that is more compact and for me more directly
readable is a clear advantage when doing maintance on code I didnt
touch for a while. Documenting the essential parameters used with some
examples and some logging helps (for me at least...).

The type of IDE you are looking for is more something like Rational
Rose (e.g. RRT?) type of tooling perhaps? There you "CAD" components
and their statebehavior and the system generates C or C++ code.

Regards,
Marco

> The maintenance thing may be true but for me that doesn't outweigh the
> clear benefits I get from using Python i.s.o. e.g. C++: the fact that
> I have much less code that is more compact and for me more directly
> readable is a clear advantage when doing maintance on code I didnt
> touch for a while. Documenting the essential parameters used with some
> examples and some logging helps (for me at least...).

Python is very easy to write. But with code grow and time passes,
things get worse. Currently I tries to reanimate one of my old
freeware project and faced it again.

> The type of IDE you are looking for is more something like Rational
> Rose (e.g. RRT?) type of tooling perhaps? There you "CAD" components
> and their statebehavior and the system generates C or C++ code.

Perhaps. I newer use Rational Rose but hear about it. But I use some
other Rational* products and they scare me a lot. So maybe Rose can be
used for "inspiration" but not much more. Actually I don't want to
put programmer in GUI-fashioned env. with a lot of buttons and
switches. I think about "classic" text-like view but that actually
"understands" that are you doing (typing) on. So your types goes in
"datastore" first and then renders back as a text representation.
Something like this.

Vladimir Ignatov

On Feb 3, 3:10*am, Vladimir Ignatov <kmis...@gmail.com> wrote:

> Finally I develop a feeling that strong instrumentation / tools can
> bring us the best of two worlds. That I am dreaming on is an absolute
> new type/class of IDE suitable for Python and potentially for other
> dynamic-type languages. Instead of current text-oriented IDEs, it
> should be a database-centric and resemble current CAD systems instead
> of being just "fancy text editor". Source text should be an output
> product of that CAD and not a "source material" itself.

That's fine so long as I can also treat the source as source, at need.

You may have just reinvented Smalltalk's Squeak editor (reputedly the
testbed for all of modern GUIs...).

Current editors suck because they can't see into the code and browse
it - unless it's so statically typed it's painful.

That's why I wrote this:

http://www.oreillynet.com/onlamp/blo...s_editors.html

Vladimir Ignatov <kmisoft@gmail.com> writes:

> Finally I develop a feeling that strong instrumentation / tools can
> bring us the best of two worlds. That I am dreaming on is an absolute
> new type/class of IDE suitable for Python and potentially for other
> dynamic-type languages. Instead of current text-oriented IDEs, it
> should be a database-centric and resemble current CAD systems instead
> of being just "fancy text editor". Source text should be an output
> product of that CAD and not a "source material" itself.

Your idea is certainly not new. Moreover, I've been using such an IDE
and implemented support for a (small) language in it:

Programs are hierarchical compositions of formulae satisfying
structural and extra-structural relationships. A program editor can
use knowledge of such relationships to detect and provide immediate
feedback about violations of them. The Synthesizer Generator is a
tool for creating such editors from language descriptions. An editor
designer specifies the desired relationships and the feedback to be
given when they are violated, as well as a user interface; from the
specification, the Synthesizer Generator creates a full-screen editor
for manipulating programs in the language.

http://portal.acm.org/citation.cfm?id=390010.808247

It might be a good start to read as much as possible on the Synthesizer
Generator if you want to write such an IDE for Python. I am sure you
could write such an IDE in the Synthesizer Generator, but I have no idea
if it's still available. And back when I used it (at university) one had
to pay for it, and most likely was closed source as well.

See also: http://www.google.com/search?q=synthesizer%20generator

--
John Bokma j3b

Hacking & Hiking in Mexico - http://johnbokma.com/
http://castleamber.com/ - Perl & Python Development

On Wed, 2010-02-03 at 10:05 -0800, Phlip wrote:
> On Feb 3, 3:10 am, Vladimir Ignatov <kmis...@gmail.com> wrote:
> > Finally I develop a feeling that strong instrumentation / tools can
> > bring us the best of two worlds. That I am dreaming on is an absolute
> > new type/class of IDE suitable for Python and potentially for other
> > dynamic-type languages. Instead of current text-oriented IDEs, it
> > should be a database-centric and resemble current CAD systems instead
> > of being just "fancy text editor". Source text should be an output
> > product of that CAD and not a "source material" itself.
> That's fine so long as I can also treat the source as source, at need.
> You may have just reinvented Smalltalk's Squeak editor (reputedly the
> testbed for all of modern GUIs...).
> Current editors suck because they can't see into the code and browse
> it - unless it's so statically typed it's painful.

? I edit Python in MonoDevelop 2.2; and I can browse my file,
classes, etc... So I don't know what you mean by "can't see into the
code". It works pretty well.

Of course it can't tell that I've set x = {an integer}, as that only
happens at runtime.

> That's why I wrote this:
> http://www.oreillynet.com/onlamp/blo...s_editors.html

On Feb 3, 10:57*am, Adam Tauno Williams <awill...@opengroupware.us>
wrote:

> > Current editors suck because they can't see into the code and browse
> > it - unless it's so statically typed it's painful.
>
> ? *I edit Python in MonoDevelop *2.2; *and I can browse my file,
> classes, etc... *So I don't know what you mean by "can't see into the
> code". *It works pretty well.
>
> Of course it can't tell that I've set x = {an integer}, as that only
> happens at runtime.
>
> > That's why I wrote this:
> > http://www.oreillynet.com/onlamp/blo...s_editors.html

You just said that your code browsing "works pretty well, except when
it doesn't".

Hence my blog entry. If your editor analyzed your code at runtime,
instead of just static analysis, then it could see that x = an
integer, or an object, no matter how dynamic your language.

--
Phlip
http://zeekland.zeroplayer.com/Uncle_Wiggilys_Travels/1

Phlip <phlip2005@gmail.com> writes:

> On Feb 3, 10:57Â*am, Adam Tauno Williams <awill...@opengroupware.us>
> wrote:
>
>> > Current editors suck because they can't see into the code and browse
>> > it - unless it's so statically typed it's painful.
>>
>> ? Â*I edit Python in MonoDevelop Â*2.2; Â*and I can browse my file,
>> classes, etc... Â*So I don't know what you mean by "can't see into the
>> code". Â*It works pretty well.
>>
>> Of course it can't tell that I've set x = {an integer}, as that only
>> happens at runtime.
>>
>> > That's why I wrote this:
>> > http://www.oreillynet.com/onlamp/blo...s_editors.html
>
> You just said that your code browsing "works pretty well, except when
> it doesn't".
>
> Hence my blog entry. If your editor analyzed your code at runtime,
> instead of just static analysis, then it could see that x = an
> integer, or an object, no matter how dynamic your language.

In Perl:

my $x = ( 5, "hello", sub {}, [], {} )[ int rand 5 ];

what's $x? The answer is: it depends.

Moreover, even if your editor analyzes your code at runtime (which is
certainly not always desirable) it might not be able to find out what
the type is of x, simply because it would take too much time to find it
out. (It sounds like you want an editor that solves the halting problem
;-) )

I agree with you that to /some extent/ and editor can do analyses, if it
does compilation as well (and even runs the code, but the latter is not
always desirable). I mentioned the Synthesizer Generator before, which
can do compilation on the fly, if you implement it (or if it has been
implemented for the language you edit with it). I've written a very
simple assembler in it, ages ago, which did assembling on the fly.

--
John Bokma j3b

Hacking & Hiking in Mexico - http://johnbokma.com/
http://castleamber.com/ - Perl & Python Development

Vladimir Ignatov wrote:
> dynamic-type languages. Instead of current text-oriented IDEs, it
> should be a database-centric and resemble current CAD systems instead
> of being just "fancy text editor". Source text should be an output
> product of that CAD and not a "source material" itself.

can you sketch an example/use case more concretely?


Robert

John Bokma wrote:

> my $x = ( 5, "hello", sub {}, [], {} )[ int rand 5 ];
>
> what's $x? The answer is: it depends.

That's why my blog post advocated (as usual for me) developer tests.
Then you either mock the rand, like all developers should, or you get
what you pay for, and Principle of Least Surprise still applies...

Over the past decade, teams discovered that developer tests more than
made up for the lack of rigor in dynamic languages. A dynamic language
with tests can be more productive than a static language, even with
its runtime type checks AND with its tests.

However, our editors must catch up to us. When I test, I am statically
declaring a set of types, even if the language would prefer to
dynamically fling them hither and yon. We should leverage that.

--
Phlip

Phlip <phlip2005@gmail.com> writes:

> John Bokma wrote:
>
>> my $x = ( 5, "hello", sub {}, [], {} )[ int rand 5 ];
>>
>> what's $x? The answer is: it depends.
>
> That's why my blog post advocated (as usual for me) developer tests.
> Then you either mock the rand, like all developers should, or you get
> what you pay for, and Principle of Least Surprise still applies...

Yup, I agree with you that (to some extent) an IDE should be able to
determine types, especially if programmers don't reuse variables, like
(again Perl):

my $result = .... # int
:
:
if ( ... ) {

$result = .... # string
}

# $result can be still an int, or either a string, depending on the
# test.

> Over the past decade, teams discovered that developer tests more than
> made up for the lack of rigor in dynamic languages. A dynamic language
> with tests can be more productive than a static language, even with
> its runtime type checks AND with its tests.

Yup, this matches up with my experience. I can't recall that I ever
bumped into an issue in Perl (the dynamic language I've been using
the most for the past years). Not saying that it hasn't happened, but I
just can't recall. Probably also the reason why a "new" language I am
learning is also dynamic: Python ;-)

> However, our editors must catch up to us. When I test, I am statically
> declaring a set of types, even if the language would prefer to
> dynamically fling them hither and yon. We should leverage that.

I am all for testing, but it should IMO not get into the way. I am quite
happy with Emacs as an editor (I "recently" switched), it satisfies most
(if not all) of the items on the check list.

--
John Bokma j3b

Hacking & Hiking in Mexico - http://johnbokma.com/
http://castleamber.com/ - Perl & Python Development

Robert <no-spam@non-existing.invalid> writes:

> Vladimir Ignatov wrote:
>> dynamic-type languages. Instead of current text-oriented IDEs, it
>> should be a database-centric and resemble current CAD systems instead
>> of being just "fancy text editor". Source text should be an output
>> product of that CAD and not a "source material" itself.
>
> can you sketch an example/use case more concretely?

I guess Vladimir means what's called a structure editor. The (by me)
aforementioned Synthesizer Generator is an example of such an editor
(environment).

http://en.wikipedia.org/wiki/Structure_editor
http://portal.acm.org/citation.cfm?doid=358746.358755

--
John Bokma j3b

Hacking & Hiking in Mexico - http://johnbokma.com/
http://castleamber.com/ - Perl & Python Development

> can you sketch an example/use case more concretely?

Sorry, I don't have anything written down. I just have some rough idea
of implementation and some concrete features I would like to see in
such system. For example:

1) Instant refactoring. No more needs for manual
search/inspect/rename. Since system knows exactly that is going on,
the refactoring will be fully automatic.
2) Show "xref table" for each function. How often this function used?
Where is it used? (code snippets of calls) What functionality is
supported by this function?
3) Extended statistics. How many objects this object/function
interacts with? Popular functions, dead/unused functions.
4) Code smell detector - too long functions, too much interaction with
other objects, global objects, etc.
....

Vladimir Ignatov