It's common to read that the concatenation method join() is much top other techniques in Python (such as + or += ). Starting from this point, I ask some questions:
join() do so fast? join() would not be welcome? join() for string concatenation in Python? References
There is a problem that one of the creators of this site (SO) calls Shlemiel the painter's algorithm .
A painter is painting the passing line of a highway. It starts off very well, with high productivity. But each day he produces less, until his work becomes unfeasible. This occurs because it holds the paint can in a fixed place, so it paints a portion of the strip and has to return to the starting point to wet the brush. Each day it is further away from the can and takes longer on the way than in painting.
Imaginethispatterngrowingtens,hundreds,orthousandsoftimes.Quicklyunfeasible.
Thisisoftenthecasewithconcatenationsofdatacollections,especiallystring.Asitgrowsanddoesnotfitintothespacethatexistedforthepreviousversion,anewallocationneedstobemadetosupportthefullsizeofthenewversion.Andyouhavetodeallocatetheoldonethathasbecomerubbish.Anditfragmentsthememory.Allthisiscostoftime.Insomelanguagesthesituationisworsesinceachangethatdoesnotevenbreakthecurrentsizelimitalreadycausesareallocationforagoodcause.ThisisthecasewithPython.
Therightwayistofigureoutthefinalsize,oratleastanapproximationofit,andallocateeverythingyouneed,youcanputthetextsinthiscrazyarea.Obviouslyitneedstobedoneinastructurethatacceptsthetexttobechanged,whichisnotthecaseoftypestringthatisimmutable,thatis,anychangegeneratesanewobject.
join()doesexactlywhatIdescribed.Itdiscoversthetotalsizeneeded-takingthesizesofallstringsthatwillbeconcatenated-allocatesallnecessaryspaceandthrowsthetextsinthatspacethatisnotyetastring.Attheenditturnsthisintostring.Therethecostisequivalenttothetotalsizeofthetextwhichismuchshorterthanwalkingalloveragainineachconcatenation.
Notethatforafewconcatenations,typicallyupto4,theconcatenationmayperformevenbetterthanjoin().Ofcourseinsuchasmallvolume,whichisfaster.
Ofcourse,join()isnottheonlywaytodothis.Youcandoitmanuallyifyouneedsomethingalittlemorecomplexthanjoin()doesnotmeet.Maybeusinga bytearray * or a default list that are mutable (help, but not great because it may need new allocations, although minimized, does not need every change, depends on the skill of the programmer).
The Python page also shows you how to use %s to get similar results. Formatting occurs in a function that manipulates everything in another data structure and only at the end that the final string is generated.
Some people like to use StringIO to take care of this.
I answered this in more detail for other languages like Java . And also C # .
From the article: Efficient String Concatenation
Method 1 (concatenation)
def method1():
out_str = ''
for num in xrange(loop_count):
out_str += 'num'
return out_str
Method 4 (join)
def method4():
str_list = []
for num in xrange(loop_count):
str_list.append('num')
return ''.join(str_list)
Method4(join)issignificantlyfasterthanconcatenation.
Thisisbecausestringsareimmutable,thatis,theycannotbechanged.To"change" one, you need to create a new representation (a concatenation of the two) and then destroy the old strings. The join is faster because Python is able to optimize this process.
The text Python: string concatenation VS list join is also very interesting and goes in the source code of the CPython implementation find out the answer:
When using the join method, Python allocates memory for the final string only once; but if you concatenate multiple strings in succession, Python has to allocate new memory for each concatenation. Guess what's fastest? ;)
That is, this is identical in terms of performance:
final_str = 'abc' + 'def'
final_str = ''.join('abc', 'def') # não há diferença de desempenho
If you concatenate more than two strings, the join will be faster:
final_str = 'abc' + 'def' + 'ghi' # aqui é realizado duas operações sucessivas
final_str = ''.join('abc', 'def', 'ghi') # aqui é realizado uma só