c# - How can I learn to plan and reason about solutions to non-trivial programming problems? -

this question posted in attempt provide tools , strategies learning, programmer @ similar level asker of https://stackoverflow.com/revisions/18104693/1

the questioner in specific case describes themself "slightly past newbie stage of learning visual c#" "an administrator not professional developer". largely self-taught programmer , not have formal experience of development work or techniques. however, have "written few respectably complex utilities (for level anyway) using web, ftp, file manip, audio, toyed around graphics , other windows forms methods". question based on own interpretation of need:

what should programmer do, has ability write programs , research, learn , use libraries, programming tools , languages, become capable of coming with, evaluating, experimenting with, choosing , implementing complete solutions programming problems?

many (most) of on either professional developers or prolific hobbyist developers, opposed knows how program. question asking how learn things every day. however, i'm not looking answers giving career advice, rather answers giving specific knowledge , tools (both tools learning , technological tools) can approach problems in way.

for convenience, give original asker's example problem, might discuss wider problem more concretely, since can refer same example.

the specific problem: through years of converting, faxing , emailing our company has created quite mess of few shared directories. learning & practice, thought write program reads directory of files looks similar following:

bkupsalesreportjan2011(txt).zip bkupsalesreportjan2011(pdf).zip bkupsalesreportjan2011(doc).zip bkupsalesreportfeb2011(txt).zip bkupsalesreportmar2011(doc).zip bkupsalesreportmar2011(pdf).zip goes on few hundred more files... 

after reading file list, want save 1 copy of each report based on list of file types (in order of priority). wanted keep pdfs , delete duplicates. if there no pdfs keep docs , lastly keep txts. simple example here feel unsure do.

the questions: add each group of named files list of own , string comparisons results , delete immediately? add deletable files list , delete when entire directory has been processed? there way compare them while reading directory? better create small database (which know little about) , unfamiliar database commands results? there buried option in windows forms need research magically me? should open source module, code or plugin takes care of me? should bug great folks @ stackoverflow.com horribly simple dilemma?

the directions in need develop skills follows:

1. you need deepen knowledge of technologies use.
2. you need learn examples of good, great , bad code.
3. you need consider context in code used.

the first 2 steps essential , take being adequate programmer one. third step is, argue, separates programmer developer. how important depends on need work , how want develop skills. sysadmin, example, programmer, doesn't need developer.

deepening knowledge

this means going having 'black box' understanding of programming languages, programs, libraries , systems being able how work. ideally programmers should have knowledge of type every level of our systems, electricity , magnetism, through cpu , computer architecture, operating systems theory (what files , processes, 2 great os abstractions, correspond to), algorithms , cs math, networking, graphics , user interface stuff, code behind compilers, common programs browsers , editors, , other tools. in practice no-one can know enough of this, have guided both what's interesting , useful you.

in example given, of following might have been helpful know more about: file or directory on disk represents , happens when move or delete it; how zip compression works; stored in various file formats , how. can solve problem without of these, need know of them sure you're not making task harder is, or risking form of data loss.

one important tool queue of things learn about. regexes mentioned in answer original question. seems happen on stackoverflow, can frustrating contributors, asks question regexes answer. because don't understand regexes, choose inferior solution rather learn them. little later asking question regexes provide solution. yet again prefer less recommended learn new. come , ask complexities of workaround solution, have avoided spending 10 minutes learning regexes.

the queue works this: first see don't know mentioned in on so, in tutorial, or in piece of code, can add list. third time see have add it. find basic on each of these things , read until understand them. if reading throws more things don't understand, add them list.

another rule make stop reading find term don't have any understanding of. thing , make sure understand first. wikipedia friend here.

you find things need know have habit of popping up. 'algorithms' great example - central cs , software. here problem there material. there literally thousands of books on algorithms, including phd-level research, etc. word use in lots of different contexts - algorithmic trading, 'google's algorithm' or 'netflix algorithm', mathematical algorithms, etc. looking @ wikipedia page 'algorithm', first paragraph clear , makes sense, second par comprehensible, contains things don't need know, , third par full of technical detail irrelevant beginner. reading whole article somewhere in between impossible , waste of time.

so tool; variety of sources. if difficult, find else, read first. decide whether need read first document, else easier, or nothing else, depending on know. try , learn minimum necessary progress, go things in more depth later. first time learn regexes, learn how use 1 in code. second time, @ list of special characters , try , use ones don't know. third time, learn optimising them run faster. fourth time, find out theory behind them.

another one: pick important and/or interesting parts. if go section 'examples' of https://en.wikipedia.org/wiki/algorithm#examples , first example great starting place. have algorithm, description of algorithm, , pseudocode (note self: pseudocode) implementation. can understand of them? graphic visualization of quicksort. quicksort same algorithm in text? can understand graphic? have added quicksort list of things up? second example detailed yet accessible explanation of 1 of important algorithms ever, euclid's algorithm. if spend hour or evening, , end still not knowing quite algorithm is, knowing quicksort , euclid's algorithm are, haven't wasted time.

another essential tool here programming scratch pad: way of writing , testing code quickly. having read quicksort or euclid's algorithm, can implement them yourself? can try other method solve same problem? want try , make animated gif quicksort graphic? having learnt regex is, can write few , test if think do. c# programmer, options include: tool linqpad running few lines of c# instantly (there online ones); keeping templates around make executables test simple things; learning language python comes tools instantly evaluate code snippets.

learning example

learning example really, important. programmers have done it, yet many books , college course ignore it. don't learn example snippets in books , on so, need real code.

when bill gates learning program, had fish printouts out of dumpster @ college learn systems programming. resources available limitless - there whole server , desktop systems can read every single piece of installed code. gives new problem of deciding which code at.

you can go searching on github or sourceforge things interested in , try run them. downside of might not code - , might not able tell code bad.

looking @ tools, small tools, have community acceptance way. there various common .net libraries simple things , open source. rather recommend some, it's best use 1 come across because want call code. make sure know how use library before try study source. if use linux or libre operating system, pick 1 of tools use everyday study. if don't, install open source , use bit before trying study code.

the basic thing take single file , read until understand does. involves choosing couple of files first discard because abstract/too complex/too difficult. (don't scared find whole different project if don't make headway. code written , commented understood outsiders , isn't.) involve amount of straight reading (maybe on paper can make notes), looking of things don't understand, , reading in ide can flick , forth between function calls.

the program study should small , simple possible. if try @ firefox code, have trawl through directories , interfaces before finding bit comprehensible you.

free software can hassle build because of dependencies. don't have able compile study it, if want know how little bit of something. make difference, because able change things , see happens. @ least, able break it, try , understand why broke.

longer answers on stackoverflow can useful, if don't @ them extract solution particular problem. if have @ least complete function, preferably class, try examine 'style' code written. deeper , more interesting formatting. functions shorter or bigger functions write? there more or fewer intermediate variables? built-in or third-party libraries used don't know about? (add them queue of things up) understand how generics, extension methods, operators, containers or linq used in answer? use comments way do? comments add?

the other nice thing , similar if don't understand why poster did way can ask them. people flattered read code carefully. if think can simplify or correct code, so. if got wrong, you'll explanation of why works specific way, add value others might make same mistake you.

read code other languages. might not able write hello world in python, c, or scheme, can understand it? useful skill find next language want learn. http://rosettacode.org unique , fascinating resource.

the holy grail of reading code find actual working code has been commented extensively and/or annotated in separate document, in order read learning programmers. there isn't lot of this. luckily, there great exception - c# god @jonskeet has written 'reimplementing linq objects' series of blog posts explains linq example , shows how well-designed, robust code planned , made.

when you're better coder, able @ bad code , see why bad. great learning tool because there lot of bad code in world! again, start - if asking piece of code, have written improve.

coding in context

going example. if in workplace asked me write this, first question - "is going more once?"

suppose answer no, , believe answer. write reasonably gets job done. if little slow, shouldn't matter me, long practical run it. rather worry safety of files, might backup whole directory, run program, , check nothing got trashed. if did, restore backup , start again.

suppose answer yes. or suppose answer no, suspect in year, want me write same thing again (lesson 1 - users don't know need.) i'm software developer, , need think lot of things before write anything.

who run code? knows doing? interface comfortable with? how know code has run properly? trick imagine being user, doesn't know or care language, design or background, , try , figure out think when program works or doesn't work in various ways. if you user, still need imagine being in different context. when file server isn't working because disk full, , want run tool, won't want open visual studio change things, or figure out how works. want designed work way want too.

what different ways run? inputs or parameters provided @ runtime? how provided? how user know how provide them? last question touched on huge topic developers - documentation. our instinct when writing small script don't want write long manual go it. program can example display list of options. should always, when asked, or when options wrong? there no code doesn't need comments, likewise there should no programs don't need documentation.

what different ways program complete? go wrong? of these errors prevent doing @ all? how communicate errors? log warnings , 'near-errors'? log normal behavior? important design technique 'fail-safe' - misused expression. doesn't mean safe failure, means 'fails in safe way'. in example, means if there error or uncertainty, don't delete anything.

stop being programmer while , tester. (joel on software, full of interesting , great stuff development, talks testers here http://www.joelonsoftware.com/articles/fog0000000067.html). try , break program. give wrong inputs, try , find errors doesn't report properly, try , make things shouldn't (like delete wrong files). have try , think of possibly go wrong, sysadmins at. errors don't find ones found when happen in production.

now thinking designing software, rather writing program, need think future. when requirements change (lesson 2 - requirements change), going adapt old version? old version going still used? new version contain functionality of old one?

one thing going have make behind-the-scenes changes don't affect how program works. other thing going have change functionality. in first case have make sure know how work, including behavior users relying on. in second case have able communicate changes users. these tasks eg. microsoft dedicate half development resources to.

get git , use checkouts, branches, merges own projects. learning use understand how developers think , work.

here can learn example too. remember github projects weren't enough learn code from? ones popular enough continuously developed getting better, , process of getting better important code itself. clone repository , follow patches , changelogs. learning how deal these in git, following how project progresses, false starts , future goals. (lesson 3 - projects 'going rewritten ground in next version')

any reasonably big open source project has mailing list archive. first sight it's full of chitchat , repetition, real work developers do. go through different bugs, features , designs again , again thrash out best way of doing things. arguably making decision hard part compared writing actual code. in these discussions trying find important edge cases affect design, trying predict performance depend on, trying figure out changes might need made in future. answer decisions agonizing over, whether use third-party libraries or not, can found here. decisions don't apply you, way decisions reached should studying.

go being tester , see if can break else's code. open source projects can surprisingly robust. , find out why so. how parsing options bad options don't make code fall on yours does? checking errors? how exceptions handled? else done internally signal 1 part of code can't work right?

when you're ready, want patch , send merged in. popular open source tool, won't ready until understand architecture of project, coding style, , direction maintainers taking it. hobby project on github, can send more tentative things. once patch gets accepted or rejected, try , understand why. can rewrite fit in better? why maintainers want things differently? how did end incorporating suggestion (if @ all)?

finally

don't try of @ once. make list of things want do, , come it. don't worry things start , don't finish, long finish something. , if find you're forcing learn find excruciatingly boring, try moving onto else. because it's fun!