Create directories and modify files with Python

Hi,

I would like to automate some simple tasks I'm doing by hand. Given a text file 
foobar.fo:

073 1.819
085 2.132
100 2.456
115 2.789

I need to create the directories 073, 085, 100, 115, and copy in each directory a modified version of the text file input.in:

.
.
.
foo = 1.5 ! edit this value 
.
.
.
bar = 1.5 ! this one, too
.
.
.

Tthe modification consists in substituting the number in the above lines with the value associated to the directory in the file foobar.fo. Thus, the input.in file in the directory 100 will be:

.
.
.
foo = 2.456 ! edit this value 
.
.
.
bar = 2.456 ! this one, too
.
.
.

At first, I tried to write a bash script to do this. However, when and if the script will work, I'll probably want to add more features to automate some other tasks. So I thought about using some other language, to have a more flexible and mantainable code. I've been told that both Python and perl are well suited for such tasks, but unfortunately I know neither of them. Can you show me how to write the script in Python? Thanks,

Best Regards

Sergio Rossi

On 01/05/2012 00:24,  wrote:
> Hi,
>
> I would like to automate some simple tasks I'm doing by hand. Given a text file
> foobar.fo:
>
> 073 1.819
> 085 2.132
> 100 2.456
> 115 2.789
>
> I need to create the directories 073, 085, 100, 115, and copy in each directory a modified version of the text file input.in:
>
> .
> .
> .
> foo = 1.5 ! edit this value
> .
> .
> .
> bar = 1.5 ! this one, too
> .
> .
> .
>
> Tthe modification consists in substituting the number in the above lines with the value associated to the directory in the file foobar.fo. Thus, the input.in file in the directory 100 will be:
>
> .
> .
> .
> foo = 2.456 ! edit this value
> .
> .
> .
> bar = 2.456 ! this one, too
> .
> .
> .
>
> At first, I tried to write a bash script to do this. However, when and if the script will work, I'll probably want to add more features to automate some other tasks. So I thought about using some other language, to have a more flexible and mantainable code. I've been told that both Python and perl are well suited for such tasks, but unfortunately I know neither of them. Can you show me how to write the script in Python? Thanks,
>
> Best Regards
>
> Sergio Rossi
>
>
>
>
>Please show us the code you've written so far, including full traceback 
for any errors.  Let's face it, this is similar to the question you 
posted some eight hours ago give or take.  For more data on the problems 
with time please see this amongst others 
http://thread.gmane.org/gmane.comp.python.dev...-- 
Cheers.

Mark Lawrence.

On 1-5-2012 1:24,  wrote:
> Hi, 0 I would like to automate some simple tasks I'm doing by hand. Given a text file
>  foobar.fo:[...]> At first, I tried to write a bash script to do this. However, when and if the script
> will work, I'll probably want to add more features to automate some other tasks. So I
> thought about using some other language, to have a more flexible and mantainable
> code. I've been told that both Python and perl are well suited for such tasks, but
> unfortunately I know neither of them. Can you show me how to write the script in
> Python? Thanks,Err.. if you don't know Python, why do you think a Python script will be more flexible
and maintainable for you?

But if you really want to go this way (and hey, why not) then first you'll have to learn
some basic Python. A good resource for this might be: http://learnpythonthehardway.org/

Focus on file input and output, string manipulation, and look in the os module for stuff
to help scanning directories (such as os.walk).

Irmen

Il giorno martedì 1 maggio 2012 01:57:12 UTC+2, Irmen de Jong ha scritto:
> > Hi, 0 I would like to automate some simple tasks I'm doing by hand. Given a text file
> >  foobar.fo:
> 
> [...]
> 
> > At first, I tried to write a bash script to do this. However, when and if the script
> > will work, I'll probably want to add more features to automate some other tasks. So I
> > thought about using some other language, to have a more flexible and mantainable
> > code. I've been told that both Python and perl are well suited for such tasks, but
> > unfortunately I know neither of them. Can you show me how to write the script in
> > Python? Thanks,
> 
> Err.. if you don't know Python, why do you think a Python script will be more flexible
> and maintainable for you?
>Eheh :) good question. The point is that, when in the past I chose to use only languages that I know, I always ended up with one of these two solutions:

1) write a "simple" shell script, which three years from now will make me curse in Sumeric because I can't make heads or tails of it. Been there, done that.

2) with time, the shell script may become larger and require some mathematics. For example, the second column of foobar.fo consists of mass flows, which today I get from another code, but that I could compute in the script. At this point, I go for a Fortran/C code, which takes me longer to write.

Now I'd like to try another road. The problem is that each time I stumble upon a good chance to learn Python, I never have enough time to learn the language from zero, and I end up with one of two solutions above. If you can provide me with a working solution or at least some code to start with, I should be able to solve my immediate problem, and then have time to learn some more (that's how I learned shell scripting). The (possibly wrong) underlying assumption is that the problem is simple enough for one of you Python experts to solve easily, without wasting too much of his/her time. Anyway, I'm willing to pay a reasonable fee for help, provided you have Paypal.> But if you really want to go this way (and hey, why not) then first you'll have to learn
> some basic Python. A good resource for this might be: http://learnpythonthehardway.org/Ehm...name's not exactly inspiring for a newbie who's short on time :)> 
> Focus on file input and output, string manipulation, and look in the os module for stuff
> to help scanning directories (such as os.walk).
> 
> IrmenThanks for the directions. By the way, can you see my post in Google Groups? I'm not able to, and I don't know why. 

Sergio

 wrote:

> At this point, I go for a Fortran/C code, which takes me longer to write.If you already have a basic programming knowledge the tutorial that comes 
with Python should be an excellent starting point:

http://docs.python.org/py3k/tutorial/index.ht...> Now I'd like to try another road. The problem is that each time I stumble
> upon a good chance to learn Python, I never have enough time to learn the
> language from zero, and I end up with one of two solutions above. If you
> can provide me with a working solution or at least some code to start
> with, I should be able to solve my immediate problem, and then have time
> to learn some more (that's how I learned shell scripting). The (possibly
> wrong) underlying assumption is that the problem is simple enough for one
> of you Python experts to solve easily, without wasting too much of his/her
> time. Anyway, I'm willing to pay a reasonable fee for help, provided you
> have Paypal.While some people here might welcome a few extra bucks that's generally not 
the reason we are posting here. The idea is more about peers helping peers 
and sometimes learning something useful for themselves.

In that spirit I'd like to make an alternative offer: put some effort into 
the job yourself, write a solution in bash and post it here. I (or someone 
else) will then help you translate it into basic Python. That will typically 
attract others who know how to make it shorter, simpler, or more general (or 
to turn it into a showcase for Python's more advanced features or their 
favourite programming paradigm). You'll end up with a usable starting point 
for further endeavours into the language, no money involved.

On Tue, May 1, 2012 at 9:45 PM, Peter Otten  wrote:
> In that spirit I'd like to make an alternative offer: put some effort into
> the job yourself, write a solution in bash and post it here. I (or someone
> else) will then help you translate it into basic Python. That will typically
> attract others who know how to make it shorter, simpler, or more general (or
> to turn it into a showcase for Python's more advanced features or their
> favourite programming paradigm). You'll end up with a usable starting point
> for further endeavours into the language, no money involved.Seconded, and I'd recommend putting this into a FAQ. Dollars aren't
the currency here, help and coding time are.

Regarding the original problem: I would recommend you dedicate just
one hour to learning Python. Most people should be able to find that
much time, I think! As a programmer, you should be able to pick up
most of Python's basics in an hour, using its own tutorial as Peter
Otten posted ( http://docs.python.org/py3k/tutorial/index.ht... );
after that, you can probably write your script, or at very least will
know which aspects of the language you love and which aspects you
hate!

ChrisA

On 04/30/2012 05:24 PM,  wrote:
> Hi,
>
> I would like to automate some simple tasks I'm doing by hand. Given a text file
> foobar.fo:
>
> 073 1.819
> 085 2.132
> 100 2.456
> 115 2.789
>
> I need to create the directories 073, 085, 100, 115, and copy in each directory a modified version of the text file input.in:
>
> .
> .
> .
> foo = 1.5 ! edit this value
> .
> .
> .
> bar = 1.5 ! this one, too
> .
> .
> .
>
> Tthe modification consists in substituting the number in the above lines with the value associated to the directory in the file foobar.fo. Thus, the input.in file in the directory 100 will be:
>
> .
> .
> .
> foo = 2.456 ! edit this value
> .
> .
> .
> bar = 2.456 ! this one, too
> .
> .
> .
>
> At first, I tried to write a bash script to do this. However, when and if the script will work, I'll probably want to add more features to automate some other tasks. So I thought about using some other language, to have a more flexible and mantainable code. I've been told that both Python and perl are well suited for such tasks, but unfortunately I know neither of them. Can you show me how to write the script in Python? Thanks,Perhaps something like this will get you started?  To
keep things simple (since this is illustrative code)
there is little parameterization and no error handling.
Apologies if Google screws up the formatting too bad.

--------------------------------------------------------
from __future__ import print_function   #1
import os

def main():
        listf = open ('foobar.fo')
	for line in listf:
	    dirname, param = line.strip().split()     #7
	    make_directory (dirname, param)

def make_directory (dirname, param):
	os.mkdir (dirname)                             #11
	tmplf = open ("input.in")
	newf = open (dirname + '/' + 'input.in', 'w')  #13
	for line in tmplf:
	    if line.startswith ('foo = ') or line.startswith ('bar = '):  #15
		line = line.replace (' 1.5 ', ' '+param+' ')              #16
	    print (line, file=newf, end='')             #17

if __name__ == '__main__': main()    #19------------------------------------------------------------

#1: Not sure whether you're using Python 2 or 3.  I ran
 this on Python 2.7 and think it will run on Python 3 if
 you remove this line.

#7:The strip() method removes the '\n' characters from
 the end of the lines as well as any other extraneous
 leading or trailing whitespace.  The split() method
 here breaks the line into two pieces on the whitespace
 in the middle.  See
  http://docs.python.org/library/stdtypes.html#...

#11: This will create subdirectory 'dirname' relative
 to the current directory of course.  See
  http://docs.python.org/library/os.html#os.mkd...

#13: Usually, is is more portable to use os.path.join() to
 concatenate path components but since you stated you are
 on Linux (and "/" works on Windows too), creating the path
 with "/" is easier to follow in this example.  For open()
 see
  http://docs.python.org/library/functions.html...

#15: Depending on your data, you might want to use the re
 (regular expression) module here if the simple string
 substitution is not sufficient.

#16: For simplicity I just blindly replaced the " 1.5 "
 text in the string.  Depending of your files, you might
 want to parameterize this of do something more robust or
 sophisticated.

#17: Since we did not strip the trailing '\n' of the lines
 we read from "input.in", we use "end=''" to prevent
 print from adding an additional '\n'.  See
  http://docs.python.org/library/functions.html...

#19: This line is required to actually get your python
 file to do anything. :-)

Hope this gets you started.  I think you will find doing
this kind of thing in Python is much easier in the long
run than with bash scripts.

A decent resource for learning the basics of Python is
the standard Python tutorial:
  http://docs.python.org/tutorial/index.html

On 1/05/12 17:34:57,  wrote:
> from __future__ import print_function   #1
>
> ------------------------------------------------------------
> 
> #1: Not sure whether you're using Python 2 or 3.  I ran
>  this on Python 2.7 and think it will run on Python 3 if
>  you remove this line.You don't have to remove that line: Python3 will accept it.
It doesn't do anything in python3, since 'print' is a function
whether or not you include that line, but for backward
compatibility, you're still allowed to say it.

Incidentally, the same is true for all __future__ features.
For example, Python3 still accepts:

from __future__ import nested_scopes

, even though it's only really needed if you're using python
2.1, since from 2.2 onwards scopes have nested with or without
that command.

HTH,

-- HansM

On 1-5-2012 12:51,  wrote:
>> But if you really want to go this way (and hey, why not) then first
>> you'll have to learn some basic Python. A good resource for this
>> might be: http://learnpythonthehardway.org/
>
> Ehm...name's not exactly inspiring for a newbie who's short on time
> :)It's just a name. That resource is generally regarded as one of the
better books to learn Python for total beginners.

If you can program already in another language, the standard Python
tutorial might be more efficient to start from:
http://docs.python.org/tutorial/> Thanks for the directions. By the way, can you see my post in Google
> Groups? I'm not able to, and I don't know why.I don't use Google groups. I much prefer a proper news agent to read my 
usenet newsgroups.

Irmen

I'm leaving the thread because I cannot read any posts, apart from Irmen's. Anyway, I would like to publicly thank all who contributed, in particular rurpy who solved my problem (and kindly sent me a personal email, so that I could see his/her post :)

Best Regards

Sergio Rossi

On 5/1/2012 5:51 AM,  wrote:
> Il giorno martedì 1 maggio 2012 01:57:12 UTC+2, Irmen de Jong ha scritto:On 5/1/2012 5:51 AM,  wrote:
> Il giorno martedì 1 maggio 2012 01:57:12 UTC+2, Irmen de Jong ha scritto:
[snip]
>> Focus on file input and output, string manipulation, and look in the os module for stuff
>> to help scanning directories (such as os.walk).
>>
>> Irmen
>
> Thanks for the directions. By the way, can you see my post in Google Groups? I'm not able to, and I don't know why.
>
> SergioThey may have copied the Gmail idea that you never need to see anything
anything you posted yourself.

When I post anything using Google Groups, my posts usually show but
often very slowly - as in 5 minutes after I tell it to post.

Robert Miles

> > Thanks for the directions. By the way, can you see my post in Google Groups?
> I'm not able to, and I don't know why.
> >
> They may have copied the Gmail idea that you never need to see anything
> anything you posted yourself.I can see all my posts in a Gmail thread/conversation but if there are no 
replies then I would have to look in All Mail. But for "normal" email
conversations it does have my posts in there. 

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423--

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/ema....