The theory is beautiful, if it were not theory.

In practice

You can never collect all the requirements of a system at the beginning of it, because the use of the system leads users to reflect on new uses, not to mention the fact that they remember new details. p>

Agile approaches will try to sell you the idea that you can stick to the client and interactively extract the requirements by negotiating a set of "coff coff" functionality at each delivery. I find it quite questionable that a top customer will pay for iterations without knowing what it will have at the end of a given time frame, so perhaps the vast majority of contracts still remain closed-end. I also find it questionable whether the client has the willingness to participate in the project as agility advocates, but it is worth trying. In practice, customers feel that guessing what they want is part of the package they are paying for, after all, it's time for it.

Since no system starts to be developed without a minimal initial notion of scope, and since the approach of trying to collect the maximum specification from the beginning is not perfect by the dynamic nature of software engineering, and since agile approaches sell the right idea that the customer should consider software as a development service and not as a product that costs X to get ready with price set from the beginning, What do we have left?

Identify early on what are the MOST IMPORTANT features and focus on them from the beginning, because if you delay (and you probably will), it is easier for the project to survive if core functionality is implemented from the beginning

Obtain from the client the commitment to the requirements, formalizing them whenever possible and prioritizing, so that you have some negotiation currency (you will need it)

Obtain system acceptance criteria from the outset, which are more important and better define what the system actually needs to do than the initial specification. In fact, if you can focus on the criterion of acceptance as the criterion where the project and the payments end up since the beginning of the project, so much the better. Otherwise, it indicates that your customer is not sure what they want and that the project should be seen as a service and not as a product.

UPDATE - just complementing, you asked for techniques. The most important technique is to know how to listen. And after a good few years, I realized that the best book I know to teach empathy and to know how to listen is this book, which although it does not seem to be applicable to software engineering, I find it perfectly possible. link

    

I agree with Leo that the theory is different from practice, and the whole answer is very good.

Theory vs. Practice

We have to be careful.

Many theories are designed to be implemented and tested in practice. They may or may not work.

But in the case of Software Engineering, what happens in general is the inverse. Much of the theory came from the observation of practice or borrowing from other areas of knowledge, such as factory procedures or project management.

The problem is that these things do not work the same way, after all you do not develop software in the same way that you build a car on production lines or if you build a building.

Analogies

Although not perfect, many analogies with these other areas help us understand what we are doing.

Similarly, a step-by-step like the one described in the question does not consist of a fixed rule to be followed, but an abstraction of what is actually done.

In other words, it's not like a cookbook or an assembly manual where you perform each step and get the final product. Think of Software Engineering theory as an attempt to make explicit or describe what we often do without even understanding.

Example

Suppose you had an idea for an internal system for your company that will improve a particular process.

You go to the person in charge of the area and ask some questions to understand how things really work.

Then he goes back to his desk and writes a text telling what the system could do to effectively improve the process.

Then you come back to talk to the person in charge and explain your proposal, that is, how you intend to make it work easier or more effective, save effort, reduce mistakes, or anything along those lines.

Imagine that he liked the idea and tells you that he would be grateful to work with you to make it real.

At this point you go back to your desk and spend a day or two sketching your idea better, creating screenshots, perhaps modeling the database, and then putting together a small, simple prototype,

Now you call the person in charge and show them what you did. Imagine he likes it, thinks he's on the right track, and just gives some hints about some details that would look better in another way.

Finally, you would spend another one or two weeks to improve the prototype, polish the interface, put some security and make the first version available.

The person responsible then starts using with his team, checks that the result has been positive and from there he will start asking for some improvements, minor adjustments and new features to further improve the gain.

Explanation

All I described above trying not to use technical terms is exactly how a project could work using the same steps described in the question, but with an agile approach.

Note that even in the agile project you have the requirements analysis, description of the system in user stories, thinking about the objects that will make up the solution and making those objects work.

The main differences are:

• It may not be necessary to spend a lot of time doing diagrams unless this is relevant to your understanding of the problem
• In the agile process, one does not try to find out everything the system must do for ever and ever, but focus on an initial idea that solves the most critical problem, test if it works, receive feedback and iterate again, that is, go back to the first step to create a new system version better and with more functionality
• The focus is on meeting customer needs and not simply creating a computational model and beautiful diagrams

Considerations

Use Software Engineering concepts to understand the principles of software development, and then use engineering or management techniques to help improve your work.

Sometimes, when the work environment is chaotic, it is advisable to adopt some existing development model. But in general, the ideal is not to try to follow a textbook, but to adapt the available methods that are relevant to you and then try to do a more effective job.

What does not work out should simply be discarded. For example, I recommend you try to make a simple class diagram without spending much time thinking about absolutely all possible relationships and attributes. After a while you will realize how useful this diagram was and not, and in the next cycle you will have more security to decide this.