What should be in Source Control?

I am currently working on source code that is over 5Gb in size. This is mostly due to a poorly thought out folder structure, there are code files, images and Excel files all jumbled together. I think a clear distinction should be made between source code and data.

Source Code

I will define source code as anything that is written in order to compile and run the project. If it is a webpage it will be all the HTML, CSS and Javascript or any file used to produce these. I would also include any configuration files and files used to build/deploy the website or project. Anything that is compiled from your source files can safely be ignored.

Data

I would define data as anything that is added to the project during its life. So if you have an upload option, anything that is uploaded I would describe as data. The site should still function without (or very little) data.

Images

Images can fit into both groups. Any icons or images attached to the functionality of the project I would class as source code. However anything that is uploaded should be classed as Data.

Database

The database should also be classed as both. The data, anything that is inside a database table should normally be classed as data. Stored Procedures, Functions and Views are all Source Code and would benefit from version control.

Source Control != Backup

Source control is not an excuse not to backup things. Don’t just commit files to source control so you know you can restore them if you need to. Files in general in source control are there so you can see how they changed over time as the code base changed. Files in you backup are a snapshot of what the application was at a point in time and will include ALL the data.

One last point before I end. If you are hosting on a Cloud Computing platform like Azure it gives you an easy way to distinguish between Data and Code.

Anything in your

Web App = Code
Blob Storage = Data
SQL = Data/Code

Each project is unique and there will always be exceptions to these suggestions but I think this is a good goal to have. What do you think?

Getting control of a codebase

So recently I started working on a new codebase. I will be honest when I first saw it, it was a mess. Here are a few of the things I did to try and regain control.

I was given access to the source code on Visual Studio Team Services. However this consisted of a single commit 3 months ago. When I looked at what was running on the production server it was clear that changes were being made live with no regard for source control.

The first thing I did was commit everything that was running live into source control.

Next I created a SQL Server Data Tools (SSDT) project to keep track of all the database objects. Previously there was a folder with some stored procedures in it, but these did not match with what was currently running.

SQL ServerI now had in source control the current state of the website and the database, so I knew I could get things back to this state if I made some bad changes.

Lets start by looking at the website code I had. There was no solution file, the only way to look at the website was to setup my local IIS to run what was in the website folder. I could then use Visual Studio to “open” my local IIS website and attach to process to debug it.

Next I Looked at Default.aspx to see how the website worked. The majority of the website code was stored in the database stored procedures. After the tag there was a <% %> which contained a Response.Write(RunSP.RunStoredProcedure(Parameter1, Parameter2, …) command, which executed a stored procedure and the results of the stored procedure was the html code including any javascript that the webpage needed to display. I will be honest I have never seen any code like it. My guess is that the developer was secretly a DBA and wanted to make any web page changes by just changing how the stored procedures work.

This meant that the website is not going to do anything without a backup of the database running, and meant my SSDT project was going to be vital. However the database was in a bad state, it consisted of a fair few broken objects and SSDT would not build.

Using find I went through each of the broken database objects to find where in the code they were being used. Luckily most were referenced in commented out code, so I just removed all the broken database objects. The database could now be built. However there was a dependency on the users table of another database. (This was the developers solution to sharing logins between websites) As I was using SSDT I added a database dependency, problem solved for now.

Next I tried publishing my database. SQL CMD encountered a parsing error.  The reason for this was my SPs contained javascript eg $(document), SQL CMD uses $(DatabaseName) as variables for different database so it was getting itself confused.

My solution was to use Find and Replace to replace all the $ with ‘ + CHAR(36) + ‘

So I now have a SSDT project that builds and publishes but still no website project.

To get the website running from Visual Studio I started off creating a .Net 4 website project and added Entity Framework 5 and MVC 3 via nuget. I then copied all the website code into the new project, and with a bit of work I got it to build. Most of the work was relating to namespaces and referencing the correct one and moving the EF model from AppCode to a custom named folder. A bit of trial and error later I had a version of the website that could be run from Visual Studio.

I have not deployed my new version of the website as it needs further testing. No automated testing or even a smoke test checklist currently exist.

Visual Studio Team ServicesAs my source code is hosted on Visual Studio Team Services (VSTS), I can get VSTS to build each commit and check I haven’t broken the build. This is not that helpful at the moment, hopefully one day I will have automated tests that can be run here as well.

Wow, I feel like I have done loads with this code so far but there is loads more still to do. I need to understand more about the business processes behind the code with a hope to understand why some architectural decisions have been made. I want to refactor the code as much as is possible, I would like to remove much of the html/javascript from the stored procedures as I can’t see that there is any advantage to running a website like this. Please correct my if I am wrong.

LINQ

LINQWhat is LINQ?

LINQ is an acronym for Language Integrated Query, which describes where it’s used and what it does. The Language Integrated part means that LINQ is part of programming language syntax. In particular, both C# and VB are languages that ship with .NET and have LINQ capabilities.

How do I use LINQ in my C# code?

To use LINQ the first thing you need to do is add the LINQ using statement.

using System.Linq;

In your code you need a datasource, for this example I am going to use a simple array, but it can be anything eg SQL, XML etc

int[] data = new int[10] { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };

Next you need a LINQ query. (Note I know the Q in LINQ means query, so I have just written query query, if you are one of those people who hates seeing PIN number you might not like this blog post.) A LINQ query is very similar to a T-SQL query, so if like me you are good with databases this should make sense to you.

In T-SQL you can have:

SELECT num
FROM data
WHERE num = 1

In LINQ this becomes:

var query =
from num in data
where num == 1
select num;

Finally you need to do  something with the query you have written. I am just going to print the results of my query to console.

foreach (var num in query)
{
Console.Write(num);
}

What other SQL like syntax can I use?

In T-SQL you can control ordering using ORDER BY, LINQ has a similar syntax orderby

orderby num descending

In T-SQL you can use GROUP BY, to do something similar with LINQ

group num by num.Type into type
select type

This now requires a change to your foreach loop so you can list what you are grouping by and what items are in that group

foreach (var type in query)
{
Console.Write(type.Key);
foreach (var num in type)
{
Console.Write(num);
}
}

JOINS

So you thought joining tables was a SQL Server only thing. Think again you can do this in LINQ

var joinquery =
from cust in customers
join prod in products on prod.CustomerId equals cust.Id
select new { ProductName = prod.Name, CustomerName = cust.CompanyName };

Conclusion

There are loads more LINQ functionality that you can use. While writing this blog I found https://code.msdn.microsoft.com/101-LINQ-Samples-3fb9811b which has loads of examples of different queries that you can write with LINQ.

This has inspired me to use LINQ more in my code and learn more about the different queries that could be written.

10 Ways to Survive as an IT Manager

IT ManagerSo after five and a bit years of being an IT Manager here is some advice I have learned along the way in no particular order. On the whole I have enjoyed myself but it has been a real challenge at times.

1. Figure out what plates are still spinning

Being an IT Manager is all about keeping everything running all of the time. A bit like spinning 5 or 6 plates. You have plates for your servers and network infrastructure, you have plates for bespoke databases that you maintain, you have plates for your staff (including any external contractors), you have plates for any websites or apps that you develop. That is a lot of plates to keep spinning and that before you start thinking about what your boss wants you to deliver. Make sure you know what is happening with all these plates, which ones are happy, which ones are on the way to the floor and which ones you need to get the glue out and repair.

2. Make it someone else’s problem

If you can blame someone else do so. If your internet goes down it is your ISPs fault. If your website dies its your hosting company’s fault. Take responsibility for problems but if when something goes wrong you can pick up the phone and ask for help, it will make your life easier.

3. Hire good staff

Hiring poor staff wastes time and money and makes you look bad by others. Demand the highest salary band for new staff that you can afford and don’t agree to hiring anyone that you have doubts about. It is easy to bow to the pressure to get someone quickly but this will always result in worse problems in the long run. Once you have a good team do your best to keep them, and warn upper management of the problems if staff leaves (basically make it their problem not yours!).

4. Learn, Learn, Learn

You may or may not have the opportunity to go on training courses. Whatever your situation spend time learning new stuff that will benefit the company and yourself. You can learn a lot by reading online, you can petition for training from your managers, you can fund training yourself, you can ask for help from your different suppliers. The more you learn, the more you can do and the more useful you can be to the company, plus the more interesting you will find the job.

5. Say No!

Don’t be afraid to say no. You will always be asked to do the impossible and if something is impossible say so at the start. It wastes everyone’s time if you spend a lot of time trying to do the impossible. Always give your reasons for saying no, and if you always say no people will think you are unhelpful. A better way to say no is to come up with a better solution. No I can’t do it your way but here is a better solution.

6. Don’t give estimates

If you are asked how long something will take you don’t answer straight away or give an exaggerated estimate. Go away and spend some time thinking of everything that is involved before replying. There will always be something that you forgot to consider when first asked about it and looking at the different components will help plan out the work needed as well as provide an estimate.

7. Know what to tell your boss, and what not to

This is a hard one to get the balance right for. You need to tell your boss enough so that they appreciate all that you do, but too much and they will stop listening and accuse you of talking in technobabble. I have never got the balance right with this one. I have always aired on the side of not telling my boss enough, and hence they don’t realize that I saved the day on Sunday night as everything is working again on Monday. Do repeat yourself. If your server is running low on resources start asking for replacement hardware early, and increase the frequency and the panic in line with the problems it is causing.

8. Understand the problems of the business

bhrzpww6aehdx1wvrrug
Businesses need to make money. If the one you work for isn’t making enough money you will soon be looking for another. If you work for IT you will quickly start to see the problems of the business, think about what simple changes IT could make to improve things that would benefit the whole company. Some of your suggestions won’t go anywhere, but some may have a massive impact. I can think of a few changes that IT have spearheaded that I am very proud of, upgrading our internet connection, simplifying or automating processes and delivering new versions of software.

9. Ask for help

Don’t be afraid of asking for help. There are lots of places to look for help. Other departments could take more on, you could recruit extra help, you could hire external contractors. You can ask questions on support forums like ServerFault or StackOverflow, many software re-sellers or other suppliers are a good point of contact for questions about things they supply. Microsoft Support was also invaluable for a server issue.

10. Think about disasters

Write a disaster recovery plan or backup policy. Yes there will always be something more important that needs doing, but just stop for a moment to think how you would feel if everything died on your watch. The one thing you can rely on with technology is that it will fail at some point. A back of the envelope plan of action is better than no plan at all, even better is a detailed plan of what to do when each and every service you rely on fails. Plan additional services with an idea of adding extra redundancy. Always have multiple Domain Controllers, think about what data you could run from the Cloud. VMs could be replicated to the Cloud, and servers could be run from there.

Imposter Syndrome

This week I handed in my notice at a job I have had for almost ten years. In a few weeks time I will start a brand new job as a web applications developer.

This is a great achievement for me and a great chance to learn and expand my development skills.

However moments after I handed my notice in, a huge wave of uncertainty enveloped me. I have read about this and I think I was suffering from an attack of Imposter Syndrome.

Despite external evidence of their competence, those with the syndrome remain convinced that they are frauds and do not deserve the success they have achieved. Proof of success is dismissed as luck, timing, or as a result of deceiving others into thinking they are more intelligent and competent than they believe themselves to be.Imposter Syndrome

I started to convince myself I couldn’t do the job I had just accepted and that moments after I arrived on my first day I would be found out as a fraud and kicked out the door.

My logical brain was no match for the Imposter Syndrome. Having been kicked out the door of this new job, I would become penniless in no time with no employer (even my previous boss wouldn’t want me back) or recruitment agency willing to speak with me.

OK lets see if we can fight off the Imposter Syndrome and look at a possible worse case scenario. I start at my new job, it is very hard work and a month or so later I have to move on.

No employer ever would sack me on my first day. My first day will be lots of learning how they do things, looking at procedures. I have lots of experience of learning new things, looking at procedures and if I put the work in I can make this new job a success. I have lots of good ideas and useful experience that no one at my new company has.

I don’t know what the solution is to Imposter Syndrome other than to try and ignore it by thinking positively and the knowledge that I have done amazing things in the last ten years and I will do the same in the next ten.

DZone

For a while now I have been sharing some of my blog posts on the Dzone website.

DZone

The Dzone website allows users to submit links to content and I have been submitting the content I have created on this website. This is how Dzone describes themselves:

With over 1 Million members, DZone.com is one of the web’s largest communities and publishers of technical content for software professionals. Developers from all over the world come to DZone for the latest and best content to hone their skills and advance their careers.

DZone MVBWell this week I have been invited to join the MVB (Most Valuable Blogger) programme. The hope is that more of my readers will find my content from the DZone website.

The DZone team will soon start sharing my content on the DZone website and I hope this will result in even more people reading what I have to say.

If nothing else this is clearly telling me that my blog is making a difference, people are taking note of me and what I have to say. I need to keep going and keep writing articles and be more consistent.