Funky Si's Blog

Funky Si walks a half marathon

Sun, 18 Feb 2024 11:00:00 +0000

On June 8th I will walk 13 miles near Bakewell in the Peak District to raise £250 (or more) for Cancer Research. To sponsor me click fundraise.cancerresearchuk.org/page/funkysi1701 , to find out more about the event go www.cancerresearchuk.org/get-involved/find-an-event/big-hike/big-hike-peak-district

So why am I doing this?

I am a Software Developer in my day to day life, which does not involve a lot of physical activity. I have become very lazy. Doing a half marathon is completely outside of my comfort zone and is not at all a me thing to do.

However last year I realised my mental health was starting to degrade. I have a family which keeps me very busy, I am facing lots of challenges with this, other members of my family have mental health issues which is starting to have a knock on affect on me.

I mentioned some of these challenges to a friend who suggested I join a walking group which was helping people with mental health problems. I went along and to my surprise I really enjoyed myself, the people were friendly and I enjoyed the walk in the countryside. After this I went out and bought some proper walking shoes

I went to the next one, and the next one and each time I enjoyed myself. I must admit that after these walks it tired me out due to my unfit nature, but it was worth it as for a few hours I wasn’t thinking about the 101 problems that I had no solutions for.

Fast forward to about a week ago, an advert for this charity walk popped up on facebook. I took a look at it and I started thinking. The 10k was a similar distance I had done on one of my walks, so I could easily do that. What if I could raise some money for a charity and go for a fun walk? For the next few days I kept thinking about it. Eventually I booked it. My thinking was if I am going to do something for charity I need to push myself outside of my comfort zone. I would never manage a full marathon, but what about a half marathon?

So tomorrow I start training, every day or so I walk for about 40 minutes, I will probably post some updates on my charity page as no doubt I will struggle to keep going.

Please if you can think about donating, and give me some encouragement I think I am going to need it.

TrekRanks #174

Fri, 05 Jan 2024 20:00:00 +0000

Trek Ranks #174: Top 5 Ensemble Episodes

I was recently invited onto the TrekRanks podcast. I had a great time talking Trek with Jim Moorhouse and fellow guest Anika Dane. If Trek is your thing, do take a listen.

Everyone contributes in this episode of TrekRanks as we break down the “Top 5 Ensemble Episodes.” Our panel highlight those episodes in which the entire cast play important roles and help portray the group as a whole, rather than as individuals. From heavy hitters to deep cuts and everything in between, this one quite appropriately covers it all.

Listen here

Make API calls from Visual Studio or Visual Studio Code

Sun, 19 Nov 2023 22:50:00 +0000

It is that exciting time of year where Microsoft announce a new version of .Net (Version 8 this time) and Visual Studio. There have been lots of announcements which I am still digesting, but one that caught my eye was a new window in Visual Studio that shows all your API endpoints.

The new window is called Endpoint Explorer and can be added from the menu View > Other Windows > Endpoint Explorer.

If you right click on one of the endpoints you can view the code or send a request. It works with minimal APIs or standard MVC APIs. If you select send a request a .http file will be created, this is a feature that has been in Visual Studio for a while but I haven’t taken much notice.

The syntax for writing these API calls is fairly simple

GET {{Catalog.API_HostAddress}}/api/v1/catalog/items

Or something more complex, this one is a POST with a body and a header. This is how you might call APIs which require authentication. Just include a header with your auth token.

POST {{Catalog.API_HostAddress}}/api/v1/catalog/items
Accept: application/json

{
    "name": "Test",
    "description": "Test",
    "price": 1.99,
    "pictureFileName": "test.png",
    "pictureUri": "https://picsum.photos/200/300"

}

You can also set up various variables at the top of your .http files by prefixing with an @, by default you get one that defines your base URL, but you can add more.

eg @variable = value

Full docs of how you use .http files can be found here

A hyperlink above each API call in your .http file is labelled Send request. This initiates the API call and your window splits in half with the right hand side displaying the response.

The response has four tabs, a nicely json formatted response, a raw response, a headers tab and a request tab.

Now you will notice an env box underneath your file tabs, where the method dropdown is in a normal c# class file. It took me a while to figure out how to setup environments, but you can add a file that contain environment specific config, like base URL for test/prod/dev etc.

This blog post tells me about it here but the main thing is you need to add a json file called http-client.env.json and create some json like this:

  "local": {
    "Api_HostAddress": "http://localhost:44338"
  },
  "dev": {
    "Api_HostAddress": "https://dev.example.com"
  },
  "test": {
    "Api_HostAddress": "https://test.example.com"
  },
  "prod": {
    "Api_HostAddress": "https://prod.example.com"
  }

Other environment specific variables can be added in the same way. Now you can easily call your API in different environments without having to change the URL each time.

But what about VS Code?

Well it turns out that VS Code has a similar feature, but it is not enabled by default. You need to install the REST Client extension. Once installed you can create a .http file and start making API calls. The syntax is very similar to Visual Studio.

VS Code has a similar concept of environments. Go to Extensions > REST Client > Extension Settings and click on Edit in settings.json. Add the same settings we used with VS to your settings.json file. You can now switch environments from the command palette “Rest Client: Switch Environment” and select the environment you want to use.

You can make use of the response of one API call in the request of another.

### 1st API Call
# @name login
POST {{Catalog.API_HostAddress}}/api/v1/identity/login
Content-Type: application/json

{
    "user": "username",
    "password": "password"
}

### 2nd API Call
GET {{Catalog.API_HostAddress}}/api/v1/catalog/items
Authorization: Bearer {{login.response.body.token}}
Content-Type: application/json

The last feature of VS Code I want to mention is that it can make GQL calls.

POST {{Catalog.API_HostAddress}}/graphql
X-REQUEST-TYPE: GraphQL
Content-Type: application/json
Authorization: Bearer {{login.response.body.token}}

query ($name: String!, $owner: String!) {
  repository(name: $name, owner: $owner) {
    name
    fullName: nameWithOwner
    description
    diskUsage
    forkCount
    stargazers(first: 5) {
        totalCount
        nodes {
            login
            name
        }
    }
    watchers {
        totalCount
    }
  }
}

{
    "name": "vscode-restclient",
    "owner": "Huachao"
}

Sorry postman but I think I have a new favourite tool for making API calls.

Become the best Actor, Astronaut or Developer you can be!

Fri, 03 Nov 2023 01:00:00 +0000

Recently I have been reading the first few chapters of Patrick Stewart’s Memoir, Making It So: A Memoir . In it he tells the story of his early life. I find it fascinating that very early on it is quite clear he had a life goal. Become the best actor Patrick Stewart can be.

Yes, there are other things in his life but this is what motivates much of what he does. At 12 he went on an 8 day residential drama course, where he met with like minded people that encourage him. So much that he continued to meet with them years after that course finished to grow his skills as an actor. This is something I would encourage everyone to do, find your tribe, find like minded people that will encourage you to grow and learn.

After school he couldn’t see a future in acting professionally so took a job at a newspaper. However when his job interfered with his acting in amateur dramatic groups, it caused him to reassess his priorities and he began saving to go to drama school. While at drama school he continued to improve and grow his acting skills. He has a singular vision in these early years to become an actor, and within a few weeks of finishing his training he had a job as an actor at a theatre. I have got up to the point where he is about to go on a world tour with a theatre group. During his first few jobs he continues to work on his acting skills. There is no doubt in my mind that this sense of continuous improvement helped mold Patrick Stewart into the actor we know today. While I am not a actor, I often think about continuous improvement, how can I be better at what I do.

A film I saw recently has a similar tale of a singular vision. A Million Miles Away tells the story of Jose Hernandez, a man that wanted to be an Astronaut and applied to NASA 12 times (he was rejected 11 times!) He started life as a migrant farm worker, (so similar to Patrick Stewart who also grew up in poverty). He did everything in his power to achieve his goals, and didn’t give up.

Jose Hernandez, when facing failure looked at those that had got into NASA. What did they have that he didn’t? He then worked on those areas to improve himself. He didn’t give up, he kept going. He works on his fitness, he got a pilots license, he learns Russian.

So what’s my point. Have a Big Goal. It can be anything but it needs to be big. My big goal of becoming a developer was achieved many years ago, and since then I have done a fair amount of drifting through life. What I need to do is be more like Patrick Stewart and continue to improve my skills, everyday make myself a better developer. What problems am I facing, lets break them down into manageable goals and get them solved so I can move on to the next lot.

DDD East Midlands

Mon, 09 Oct 2023 00:00:00 +0000

On Saturday 7th October I attended DDD East Midlands . It is a community run conference for Developers (the D’s stand for Developer) and I had a great time, I have been to DDD North before but this was the first time going to Nottingham for a conference (I grew up in Nottingham, or a town not far from). As I have said before there is nothing quite like chatting over coffee or lunch with fellow Developers about the tech you are learning about, this was probably my favourite thing about conferences. Now on to the talks:

Dungeons, Dragons and Developers

Matt Brunt gave an excellent keynote talk, comparing the role playing game Dungeons and Dragons with working on Software projects. A good D&D game involves team work, very like a good Software Team. A team of all wizards isn’t going to get you very far, the same way a team full of developers that excel at algorithms, won’t be good at testing, talking to customers etc. A wizard that has a secret spell to defeat the Troll under the bridge, is almost as bad as a developer that hordes his knowledge and “saves the day” when things go wrong. Keeping connected with your team as you go out on your quest so no one gets lost is as important as keeping connected with your software team.

Microservice Mistakes

Matt Hunt talked about microservice mistakes he had made.

Long lived small teams
Can’t test by only interacting with the frontend
Use events as well as API calls
Need to be able to deploy independently of each other
Have a specific reason for using microservices, a problem to solve

The Art of the Bad Code Review

A 15 minute lightning talk with Martyn Kilbryde A fun talk showing some examples of bad code reviews, a few takeaways from this:

Explain what is being fixed with a link to the ticket
Give specific, actionable feedback
Balance the boy scout rule of leaving the code better than when you found it with do one thing rule
Don’t make assumptions -> have a conversation instead
Be helpful and constructive, not dismissive and negative
Use a branch per ticket, don’t tie them together
Refer to the requirements
No room for ego, code belongs to the team not an individual
Avoid overly verbose comments
Pragmatism is important
If urgent, raise additional tickets with tech debt issues
Challenge if necessary, but do so respectfully
Don’t do a piecemeal review

Git Under the covers

Another 15 minute lightning talk with Dan Clarke This looked at the inner workings of git, this was fascinating as every part points at another part which is obviously part of the reason git is so powerful. It was quite technical and I am sure some of it went over my head.

This talk could have been a blog post

Another 15 minute lightning talk with Jamie Tanna . This was a good talk about the importance of blogging. He used the phrase blogumentation, now I have heard of this before but not its name. It is where you use your blog to document what you have learnt, it can be anything a new command or a new language or technology. This talk inspired me to write this blog post, Thank you Jamie. His blog can be found at https://www.jvt.me/posts/2023/10/07/why-blog/

How to create the conditions where happy people do their best work

Paul Bailey introduced the concept of Communities of Practice. Lots of the features of a Community of Practice I have heard of or been in teams that implement. “Communities of Practice are groups of people who share a concern or passion for something they do and learn how to do it better as they interact regularly”

Building Robots for Complete Beginners

Last talk of the day was my favourite (because it was about Robots) Mark Goodwin talked about building Robots using Rasbery Pi Pico (a microcontroller board), most of his examples used python. His demo can be found at https://github.com/computerist/robot-demos This fascinated me as I have already dipped my toe into this area when playing with microbits

Picard Season 3

Thu, 23 Feb 2023 00:00:00 +0000

Last night I had a bit of a chat about season 3 of Star Trek Picard.

Before #startrek #Picard S3ep2 airs this week; @danielhuckfield Special Guest @funkysi1701 and @hitch_daniel chat about the first episode on the Nexus Nights spin-off show.https://t.co/Jyov53z7hC
— The Temporal Trek Podcast(s) (@rider_coattail) February 23, 2023

Here is the episode I recorded: TTNN Ep 36 - PIC S3ep1 Reaction Chat

Dan and Dan are joined by previous Temporal Trek guest Si Foster for a little chinwag over the return of PICARD for the final season.

Badass Bev, Starships of Thesseus, love to loathe Captain Shaw, Predictions and More.

It was great fun to record and I hope you enjoy giving it a listen., Hopefully I will be back at the end of the season to discuss the conclusion.

How much does it cost?

Sat, 04 Feb 2023 00:00:00 +0000

How much does it Cost to run my side projects? This is going to be an interesting look at my spending, hopefully it can show you can do a lot without breaking the bank!

I have a personal Azure subscription that I pay for myself. I don’t use anything connected to my work, as jobs change and I don’t want to have to shift stuff if I change employers (which happened quite recently). I like Azure, and I like learning what I can do with it, so spending small amounts on a personal subscription is not a problem for me.

This is how much Azure is costing me daily, I like to keep an eye on this chart fairly often so I can see if I get any increases, especially if I have added a new service.

This is my monthly charges over the last few months. In Sept/Oct I switched from CosmosDB to Mongo DB Atlas as my database backend. Experience has told me that my biggest charge is data storage, websites are cheap but any database or storage is what tends to cost the most.

This website you are reading this on (assuming you are not on DevTo or Hashnode) is running Hugo which runs on an Azure Static Web App, the free tier so almost free. I have two of these, a production one and a test one. Source code is on Github (another free service), and auto deploys via Github actions. I have a load of images which I have stuffed in blob storage from when I used to run a wordpress site, the cost for file storage is minimal.

I also have a service I built using Azure Functions and Static Web Apps which makes use of public APIs like (Twitter/Mastodon/GitHub etc), data for this is stored in the free tier of Mongo DB Atlas. I then use consumption tier Azure Functions and Free Static Web App to do stuff. The definition of what I have deployed is all setup in Pulumi, so I have identical environments for Dev/Test/Prod mostly because I can rather than it being business critical or getting a lot of traffic.

I currently have 240Mb of data in my free MongoDb database, I have 512 Mb max storage to play with. I have plenty of data to play with at this tier, but I may investigate other options that involve storing less or what costs might be. I previously made use of cosmosDB, which was costing me between £10-£20 per month, so not breaking the bank, but also cheaper to use MongoDB.

Interrogating the Azure Cost Analysis tool reveals that my biggest cost is file storage (67p Prod, 101p Test and 93p Dev), I may investigate this a bit and see what is using this, but its not breaking the bank so I am pretty happy.

So what else does it cost to run my things?

My domains live on CloudFlare, I recently got a bill for £8.04 for a couple of renewals for the next 12 months. But I am struggling to think what else costs me to run my digital things.

What podcasts have I been listening to - Week 2

Mon, 23 Jan 2023 00:00:00 +0000

Another week has gone by so here is a rundown of my podcasts listenings…

Hachyderm’s Kris Nova on running a Mastodon Server

Hanselminutes episode 871

A great chat about how Hachderm grew as people moved from Twitter to Mastodon. You can find me on Hachyderm https://hachyderm.io/web/@funkysi1701 Hachyderm saw amazing growth in a short space of time, so great to hear how that impacted the service, which Kris Nova provides for free in her own time.

The Shipment

The Temporal Trek Season 3 episode 58

DanDan discuss the season 3 episode The Shipment, which sees Enterprise season 3 back on track, with a proper Star Trek story. But I can’t believe Dan forgot to include the DanDan jingle its why I tune in each week.

TrekRanks Potpouri

TrekRanks episode 150

This was a great show celebrating 150 shows, and was a lucky dip into all the 149 topics that have been chosen before. Jim was caught out by the guests this week as they went above and beyond finding answers for all possible shows that could be picked. As always some great picks reminding me why I love the show.

Troy Hunt’s Weekly Update

Weekly Update 327

I like listening to what Troy has been up to in the last week. (I am a bit behind as this show came out at Christmas time) Troy talks about Twitter and Elon Musk, and the LastPass data breach. I am a big fan of 1Password for my password manager but my employer uses LastPass

Troy Vinson: Learning From the Rackspace Security Breach

Azure DevOps Podcast Episode 225

An interesting look at security and the recent rackspace incident. I used to be a big fan of Rackspace but in recent years I have been more interested in cloud offerings like Azure.

Dr. Gregory Kapfhammer wants to stop flaky tests

Hanselminutes episode 874

I am currently looking at fixing some flaky tests on a codebase I am not familiar with so this is a great podcast to listen to. The episode goes through common reactions to flaky tests, why tests get flaky and what to do about them.

What podcasts have I been listening to

Thu, 05 Jan 2023 00:00:00 +0000

I love listening to podcasts so I thought I would share what I have been listening to over the last few days.

Source Open vs Open Source and IdentityServer with Dominick Baier and Brock Allen

.Net Rocks episode 1824

The creators of the open source project Identity Server which went commercial discuss the reasons for doing so. This was a very interesting discussion on the challenges of running open source projects and ways that they can be more maintainable. I have seen Twitter arguments on both sides about various products that have done this. It is a very nuanced problem, which I don’t know much about so interesting hearing what was involved.

Growing as an Engineering Manager with Taylor Poindexter

Hanselminutes episode 871

A discussion about being an engineering manager, mentoring and growing and learning. I am not a manager and I don’t have people that report to me, however, I am a senior developer so anything that can help me improve on the leadership side of things is great. On a side note, I love how diverse the guests are on Hanselminutes, I often listen to people on this show I would never have heard of or known about if I just stuck to topics I know about.

Top 5 Scenes at a Table

TrekRanks episode 149

Star Trek for me has always been about positivity and Trek Ranks delivers this in spades. The concept, in case you don’t know, is to choose 5 episodes about the chosen topic, but it is really an excuse to talk about them! So what 5 scenes at a table would I choose?

Undiscovered Country - Dinner with the Klingons. Klingons, Shakespeare, terrible table manners what is not to love about this scene
Q Who - Scene in the observation lounge with Q and Guinan as they discuss how to respond to the newly discovered Borg
Civil Defence - Scene in Ops between Garak and Dukat while the crew hide from an automated phaser firing at them, the best bit is when Duakt tried to leave and ends up trapped with the rest of the crew
Mirror, Mirror - Scene where Bones realises that some things are the same in the mirror universe, the spot where he spilt acid
Unification Part 2 - Spock, Picard and Data have been captured by Sela, and Sela comments that she doesn’t get to write many speeches, and then Data comments that maybe she would be happy doing another job

2022 What Happened?

Sat, 31 Dec 2022 22:00:00 +0000

As 2022 draws to a close its time to do my annual look back at what happened.

I am going to talk a bit about my personal life as well as tech things, so if that is not your bag feel free to skip this post and read my more technical articles.

January - I started the year teaching my son to code .
February - I did some work on my website this year, and it is now using Hugo instead of wordpress.
March - I was guest on a Star Trek podcast , hopefully a repeat appearance in the works very soon. I also passed an AWS exam
April - Lots of work with Scratch with my boy
June - Scottish Summit
August - My boys birthday so some coding activities with his Microbit
October - I bought a second hand car and less than a week later it burst into flames, a scary event but no one hurt and everything has since been replaced/repaired.
November - I lost my job
December - Extended time with family for Christmas with a new job starting in January. I sat and narrowly missed passing AZ-204 (Developing Solutions for Azure). I attended DDD North , I attended LeedsSharp and went on a virtual tour of a Microsoft Datacenter.

I don’t know what is in store for 2023 but I am sure it will be equally interesting. The first thing to get my head around is my new job.

So onto my goals for 2023

New job and get my head around all the new stuff I need to learn
Celebrate 10 years of marriage and concentrate on family things
Pass AZ-204 - It was such a close miss that I need to retry this
More coding with my son, he got a laptop for Christmas and would be great to build on what he has learnt so far.
Blog regularly - I doubled the number of blog posts from 2021, but it would be great to keep up the momentum with regular posts of what I am up to.
I recently added an events page, so I need to go to some events. Events like DDD North and Scottish Summit are great, but so are local user groups so a mix of these would be great.

Last year I had the following goals

Video - Not used my green screen yet, would be good to make some time to use this in 2023
Conference - I attended Scottish Summit and DDD North, would be great to go to more like these this year as well
Blog More - Yeah I achieved this, but lets keep doing this.
Metrics - I have done little bits with this over the last year, but not a huge amount.
My personal brand has improved this year as my blog relaunched on the Hugo platform, and I am very happy with its look, probably more I can do.

DDD North

Mon, 05 Dec 2022 18:00:45 +0000

On Saturday 3rd December I attended DDD North . It is a community run conference for Developers (the D’s stand for Developer) and I had a fab time there. Many conferences are going online which is great, however there is nothing quite like chatting over coffee or lunch with fellow Developers about the tech you are learning about, this was probably my favourite thing about the conference.

The first talk I attended was ‘.Net Minimal APIs’ by Kevin Smith . Kevin runs .Net user groups in York and Sheffield so I have heard him speak before. He went through some of the features of minimal APIs. I have briefly experimented with them in .Net 6, however from the sound of things the tech is much improved in .Net 7, with lots of improvements, so this is something I want to give another try.

During the coffee break I managed to have a quick conversation with Nick Chapsas . Nick has a YouTube channel that my colleagues at work have been sharing during our weekly knowledge share sessions. It was great to chat with someone that I have been watching on YouTube.

The second talk was ‘When microservices go bad’ by Andy Clark . This talk was a discussion of the different ways you can use Polly , Carl Franklin of .NetRocks fame has mentioned Polly a few times on the show, so this was a good talk to explain the different ways it can be used.

The third talk was ‘Don’t put your messages in a bottle; Implement messaging patterns’ by Eldert Grootenboer , he should have been joined by Olena Borzenko however she was unable to travel (didn’t catch exactly why, I think she was from Ukraine or working in Ukraine and Visa’s from that part of the world are a problem at the moment). This was a talk all about message queues and the patterns and practises you should implement. I don’t know much about queues, having only recently started using AWS SQS messages recently so this was useful to put that work into context. The slides from this talk are available here

Lunch was next and the food was excellent. I found myself sitting with Jamie Taylor (from the Tabs and Spaces podcast), Craig Pickles , Chris Sainty and a few others. There was a fascinating discussion on Blazor, .Net and how Microsoft introduces new stuff. Really appreciate being able to listen in on such an interesting discussion.

The fourth talk was ‘Why don’t people seem to be able to diagnose problems these days?’ by Richard Fennell . This was a fascinating walk through computer history and the challenges of finding where the problem is. The slides from this talk are available here

The last talk was ‘Empathy, Sympathy, and Compassion’ by Jamie Taylor who I had met at lunch. This was a fascinating look at how we place barriers between ourselves as Developers and the users of the software. It expanded into ethics, code of conduct and doing good. The slides from this talk are available here

It was a great event so thank you DDD North for all your hard work, I look forward to the next one. I have lots to go away and think about.

Expiring Azure AD Client Secrets

Tue, 18 Oct 2022 18:00:45 +0000

This morning the client complains that the website is not working. I do some checking and I confirm that there is indeed an error.

The error they are seeing is:

An unhandled exception occurred while processing the request.
MsalServiceException: A configuration issue is preventing authentication 
- check the error message from the server for details. 
You can modify the configuration in the application registration portal. 
See https://aka.ms/msal-net-invalid-client for details. 
Original exception: AADSTS7000222: The provided client secret keys for 
app '00000000-0000-0000-0000-000000000000' are expired. 
Visit the Azure portal to create new keys for your app: https://aka.ms/NewClientSecret, 
or consider using certificate credentials for added security: https://aka.ms/certCreds.

The fix for this is relatively simple, generate a new client secret in Azure AD and update the website config to make use of it.

The problem is that this problem will reoccur in 3, 6, 12, 18 or 24 months, depending on what expiry date you set on the secret.

A suggestion from twitter is that Logic Apps and Event Grid might help solve this problem. Lets see how this might work.

Logic Apps have a consumption plan, so this kind of solution shouldn’t break the bank as will only cost per run eg daily/weekly etc

But first here is a powershell script that lists all the secrets that are due to expire in the next 30 days.

#secret expiration date filter (for example 30 days)
$LimitExpirationDays = 30

#Retrieving the list of secrets that expires in the above range of days
$SecretsToExpire = Get-AzureADApplication -All:$true | ForEach-Object {
    $app = $_
    @(
        Get-AzureADApplicationPasswordCredential -ObjectId $_.ObjectId
        Get-AzureADApplicationKeyCredential -ObjectId $_.ObjectId
    ) | Where-Object {
        $_.EndDate -lt (Get-Date).AddDays($LimitExpirationDays)
    } | ForEach-Object {
        $id = "Not set"
        if($_.CustomKeyIdentifier) {
            $id = [System.Text.Encoding]::UTF8.GetString($_.CustomKeyIdentifier)
        }
        [PSCustomObject] @{
            App = $app.DisplayName
            ObjectID = $app.ObjectId
            AppId = $app.AppId
            Type = $_.GetType().name
            KeyIdentifier = $id
            EndDate = $_.EndDate
        }
    }
}

#Printing the list of secrets that are near to expire
if($SecretsToExpire.Count -EQ 0) {
    Write-Output "No secrets found that will expire in this range"
}
else {
    Write-Output "Secrets that will expire in this range:"
    Write-Output $SecretsToExpire.Count
    Write-Output $SecretsToExpire
}

This works quite well, but I have to manually run it every so often. I have not found a way to automate this as I don’t know how to get past the MFA that is configured on my Azure AD tenant.

Now lets look at how a Logic App might help us. First of all we need somewhere to query the expiry dates, and Azure Key Vault is the ideal place to store them.

Create a Key Vault if you don’t already have one and add a new Secret. Manually add the Client Secret from Azure AD and give it a name you can identify it with. Tick the Set expiration date and save the expiry date of your secret.

Now Create a Logic App, I am going to use Consumption, so it only costs you when it runs.

Using the Logic Apps Designer, start with a Recurrence trigger. This lets you queue up your logic app on a schedule, I’ve set mine to run every 30 days.

Next run a Get Secret command from your Key Vault. Select the identifying name you saved the secret with earlier. This will retrieve the secret from the Key Vault, if you go into the settings you can hide the output as you are only really interested in the expiry date.

I am doing a low tech solution for now, and just emailing myself the expiry date. It is easy to add the expiry date of the secret to an email.

Later on I might add a bit more logic to only email myself when the expiry is less than 30 days, a reminder to create a new secret. When creating a new secret in Azure AD, I also need to add the secret and expiry date into Key Vault.

At the very least this solution will give me an early warning of when my secrets are about to expire. It is possible to get my website to read information from Key Vault directly which I may well investigate.

Upgrading to .Net 7

Sat, 01 Oct 2022 18:00:45 +0000

I have a Blazor application that was built with .Net Core 3, and I have updated it to .Net 5 and later .Net 6 when those versions of were released. In November .Net 7 is released so I have been testing it with the .Net 7 previews.

However for this update I encountered a few more issues than I remember from the other updates.

Automapper

The first issue I encountered was with Automapper. My Unit tests failed with the following error

Error Message:
   System.ArgumentException : GenericArguments[0], 'System.Char', on 'T MaxFloat[T](System.Collections.Generic.IEnumerable`1[T])' violates the constraint of type 'T'.
---- System.Security.VerificationException : Method System.Linq.Enumerable.MaxFloat: type argument 'System.Char' violates the constraint of type parameter 'T'.
  Stack Trace:
     at System.RuntimeType.ValidateGenericArguments(MemberInfo definition, RuntimeType[] genericArguments, Exception e)
   at System.Reflection.RuntimeMethodInfo.MakeGenericMethod(Type[] methodInstantiation)
   at AutoMapper.Internal.TypeDetails.<>c__DisplayClass25_1.<GetPublicNoArgExtensionMethods>b__10(MethodInfo extensionMethod)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Enumerable.ConcatIterator`1.MoveNext()
   at System.Linq.Enumerable.SelectManyIterator[TSource,TCollection,TResult](IEnumerable`1 source, Func`2 collectionSelector, Func`3 resultSelector)+MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.UnionIterator`1.GetNext()
   at System.Linq.Enumerable.UnionIterator`1.MoveNext()
   at System.Linq.Enumerable.ConcatIterator`1.MoveNext()
   at AutoMapper.Internal.TypeDetails.PossibleNames()
   at AutoMapper.Internal.TypeDetails.GetMember(String name)
   at AutoMapper.Configuration.Conventions.DefaultName.GetMatchingMemberInfo(TypeDetails sourceTypeDetails, Type destType, Type destMemberType, String nameToSearch)
   at AutoMapper.Configuration.Conventions.ParentSourceToDestinationNameMapper.GetMatchingMemberInfo(TypeDetails sourceTypeDetails, Type destType, Type destMemberType, String nameToSearch)
   at AutoMapper.Configuration.Conventions.DefaultMember.MapDestinationPropertyToSource(ProfileMap options, TypeDetails sourceTypeDetails, Type destType, Type destMemberType, String nameToSearch, List`1 resolvers, IMemberConfiguration parent, Boolean isReverseMap)
   at AutoMapper.Configuration.Conventions.MemberConfiguration.MapDestinationPropertyToSource(ProfileMap options, TypeDetails sourceType, Type destType, Type destMemberType, String nameToSearch, List`1 resolvers, Boolean isReverseMap)
   at AutoMapper.Configuration.Conventions.NameSplitMember.MapDestinationPropertyToSource(ProfileMap options, TypeDetails sourceType, Type destType, Type destMemberType, String nameToSearch, List`1 resolvers, IMemberConfiguration parent, Boolean isReverseMap)
   at AutoMapper.Configuration.Conventions.MemberConfiguration.MapDestinationPropertyToSource(ProfileMap options, TypeDetails sourceType, Type destType, Type destMemberType, String nameToSearch, List`1 resolvers, Boolean isReverseMap)
   at AutoMapper.ProfileMap.MapDestinationPropertyToSource(TypeDetails sourceTypeDetails, Type destType, Type destMemberType, String destMemberName, List`1 members, Boolean reverseNamingConventions)
   at AutoMapper.TypeMap..ctor(Type sourceType, Type destinationType, ProfileMap profile, Boolean isReverseMap)
   at AutoMapper.ProfileMap.BuildTypeMap(IGlobalConfiguration configurationProvider, ITypeMapConfiguration config)
   at AutoMapper.ProfileMap.Register(IGlobalConfiguration configurationProvider)
   at AutoMapper.MapperConfiguration.Seal()
   at AutoMapper.MapperConfiguration..ctor(MapperConfigurationExpression configurationExpression)
   at AutoMapper.MapperConfiguration..ctor(Action`1 configure)
   at System.RuntimeType.CreateInstanceDefaultCtor(Boolean publicOnly, Boolean wrapExceptions)
----- Inner Stack Trace -----
   at System.RuntimeMethodHandle.GetStubIfNeeded(RuntimeMethodHandleInternal method, RuntimeType declaringType, RuntimeType[] methodInstantiation)
   at System.Reflection.RuntimeMethodInfo.MakeGenericMethod(Type[] methodInstantiation)

At first I didn’t realise it was an issue with Automapper and I started searching for breaking changes in dotnet 7. Eventually this lead to finding that a change to .Net had created an issue with version 11 of Automapper. The following github issues explains more about what was involved in fixing this.

https://github.com/dotnet/runtime/issues/69119

https://github.com/AutoMapper/AutoMapper/pull/3999

This week version 12 of Automapper was released and it fixed this issue for me.

Code Coverage

My build includes the calculation of code coverage of my unit tests. For some reason this step was failing.

    - task: DotNetCoreCLI@2
      displayName: 'Running API Tests'
      inputs:
        command: 'test'
        arguments: '--no-restore --no-build --configuration $(buildConfiguration) --runtime win-x64 /p:CollectCoverage=true /p:CoverletOutputFormat=opencover'
        projects: 'Path To csproj file'
        nobuild: true

For .Net 6 this step would run my unit tests and include a section that looked a bit like this.

Calculating coverage result...
  Generating report 'D:\a\1\s\WhatEver\coverage.opencover.xml'

+---------------------------+--------+--------+--------+
| Module                    | Line   | Branch | Method |
+---------------------------+--------+--------+--------+
| Something                 | 50.91% | 48.77% | 65.44% |
+---------------------------+--------+--------+--------+
| Something.Core            | 41.79% | 6.92%  | 47.2%  |
+---------------------------+--------+--------+--------+
| Something.Core.Email      | 96.55% | 50%    | 100%   |
+---------------------------+--------+--------+--------+

+---------+--------+--------+--------+
|         | Line   | Branch | Method |
+---------+--------+--------+--------+
| Total   | 49.46% | 44.2%  | 53.55% |
+---------+--------+--------+--------+
| Average | 63.08% | 35.22% | 70.88% |
+---------+--------+--------+--------+

However for .Net 7 the unit tests would pass but the coverage bit above would not run, and no visible errors to help me.

Testing dotnet test locally I discovered that if I included the path and filename to the csproj file the coverage would not run, if I removed this it would work as per .Net 6. No idea why this was happening.

I did some testing with the DotNetCoreCLI@2 task, but I couldn’t get the coverage to work. However I was testing this locally via the command line so it was a simple thing to swap DotNetCoreCLI@2 for a command line step. After I did that the coverage started working again!

Here is the command line step I added

    - task: CmdLine@2
      displayName: 'Running API Tests'
      inputs:
        script: |
          "C:\Program Files\dotnet\dotnet.exe" test --logger trx --results-directory D:\a\_temp --no-restore --no-build --configuration Release --runtime win-x64 /p:CollectCoverage=true /p:CoverletOutputFormat=opencover
        workingDirectory: '$(Build.SourcesDirectory)/PathToWhereEver'

SQL Server tables with triggers

My build was all working so now time to test the application was working. But an error was being thrown if I tried to save any data to the SQL Server database.

Could not save changes because the target table has database triggers. Please configure your entity type accordingly, see https://aka.ms/efcore-docs-sqlserver-save-changes-and-triggers for more information. The target table 'Something' of the DML statement cannot have any enabled triggers if the statement contains an OUTPUT clause without INTO clause.

A nice error, as it gives me a link to the change and what to do to fix.

protected override void ConfigureConventions(ModelConfigurationBuilder configurationBuilder)
{
    configurationBuilder.Conventions.Add(_ => new BlankTriggerAddingConvention());
}

public class BlankTriggerAddingConvention : IModelFinalizingConvention
{
    public virtual void ProcessModelFinalizing(
        IConventionModelBuilder modelBuilder,
        IConventionContext<IConventionModelBuilder> context)
    {
        foreach (var entityType in modelBuilder.Metadata.GetEntityTypes())
        {
            var table = StoreObjectIdentifier.Create(entityType, StoreObjectType.Table);
            if (table != null
                && entityType.GetDeclaredTriggers().All(t => t.GetDatabaseName(table.Value) == null))
            {
                entityType.Builder.HasTrigger(table.Value.Name + "_Trigger");
            }

            foreach (var fragment in entityType.GetMappingFragments(StoreObjectType.Table))
            {
                if (entityType.GetDeclaredTriggers().All(t => t.GetDatabaseName(fragment.StoreObject) == null))
                {
                    entityType.Builder.HasTrigger(fragment.StoreObject.Name + "_Trigger");
                }
            }
        }
    }
}

Adding the following to my DBContext allowed data to be saved even if triggers existed on my database.

A longer term fix, should be to investigate if the triggers on the database are still needed and remove if not. From reading the docs, it sounds like a performance gain would be had after doing this!

Conclusion

These are the issues I have encountered so far updating to .Net 7. A few more than for .Net 5 or 6. However so far I have managed to find a solution for all issues. I expect one more release candidate, with the final version of .Net 7 coming out in November.

MakeCode and the BBC micro:bit

Sun, 21 Aug 2022 18:00:45 +0000

Programming is in my DNA, so when I had children it was obvious that I would attempt to share my passion. It has taken a while, for some reason for loops don’t interest toddlers! However my 7 year old son has got a taste for it now.

It was his birthday the other day and he got quite a few tech related things. First off he got some programming books. Get Coding! , Coding for Kids: Scratch: and Coding for Kids: Python:

He has been reading an old Javascript book, so I wanted to get him something a bit more up to date, so the Get Coding book should give him the basics of creating webpages and doing things with Javascript. He has been also been playing with Scratch for a while so the second book will definitely interest him. He also keeps mentioning Python, which I know nothing about, so even if he never reads it, I have something new to learn!

As well as the books he got a BBC micro:bit (with some accessories!) from his grandparents. He has been playing with MakeCode which features a simulator of a micro:bit, so I thought he might like having a physical device in his hands.

He has loved playing with it so far, and I have loved working with him. Already we are at the point where he knows where to find some of the blocks he needs to write the programs, and is telling me I am looking in the wrong place!

In case you are not familiar with MakeCode it is a block based programming language similar to Scratch. You drag blocks onto the screen which represent if statements or variables or loops etc. The micro:bit section of the site, simulates a BBC micro:bit and allows you to write programs for it. There is a download feature which allows you to download the program you have written and run it on a physical device. The process of downloading and running your code on the micro:bit is really straight forward. The micro:bit can be connected to your pc/laptop with a micro usb cable, which also powers the device. The device then shows up as a drive and the programs can be downloaded directly onto the device.

The BBC micro:bit is a pocket-sized computer that introduces you to how software and hardware work together. It has an LED light display, buttons, sensors and many input/output features that, when programmed, let it interact with you and your world. There is a huge list of sensors from buttons, to accelerometer, to temperature and light. It really is an amazing device.

The MakeCode website is full of help pages, tutorials and videos, so whatever level or experience you are, you will be able to get started. Also you can get a lot out of the site without having a physical device as the simulator is rather good. However me and my boy, have really enjoyed having a physical device in our hands, that we can move around the house with. We programmed up a simple light sensor, and we could move around the house finding where was dark and what happened if we moved our hands in front of it. The website allows the same thing, however moving a slider is not the same experience.

Twenty five years ago (or so) my step father was getting me into electronics, by explaining capacitors, resistors and all the other components. So it is no surprise that he has loved setting things up for my boy. He has spent a great deal of time getting components that would work with the micro:bit and 3D printing boards and connectors (to make it easy to connect things up)

Pictured are a selection of the components. We have a selection of breadboards which we haven’t tried yet, a selection of cables for connecting things up. Two battery boxes, (3V and 9V). A slider variable resistor, we experimented with using this as a volume control earlier. A board that makes it easy to connect components to the micro:bit. A joystick. A NeoPixel strip of ten colourful LEDs. A 8x8 LED Matrix for cool scrolling messages. The last two components, the NeoPixel and the Matrix require an extension to the standard controls but these are easily found on the MakeCode website.

I have featured on this page the code I helped write that uses the joystick to move a LED around the 5 x 5 grid that is on the micro:bit. To write this I started off by printing the X Co-ordinates to the screen. With the joystick in the centre position, it was about 500, when moved one way it was low teens, and the other way it was 1000+. Then I needed to figure out how to convert X, Y Co-ordinates for the joystick that could be 0 to 1000, to X, Y co-ordinates of the display that is 5 x 5. The Maths was a bit beyond my boy, but he just about understood what was going on. The clear screen step is needed so the previous position can be removed from the screen.

I am really enjoying learning about the micro:bit and seeing my boy engage with it and learn programming. I have been very impressed with his ability to retain knowledge of ifs and other programming concepts (however I am a bit biased being his father)

A brief history of Source Control

Mon, 25 Jul 2022 18:00:45 +0000

I have been thinking back to when I started using source control and some of the different tools I have used over the years.

When I was learning to code it was some time after I had learned the basics that I learnt about source control. Back when I started writing webpages I would have a folder containing all my html, css, images etc and I would FTP these up to my web server. If I made a change to a file I would reupload the file I changed, or reupload all my files to make sure I didn’t miss anything.

It was sometime later that I learnt about source control. I can’t remember exactly when, but I suspect it was after I had used it at work. (between 2006 and 2010 as a best guess, my oldest repo says it is 12 years old!)

Source control (or version control) is the practice of tracking and managing changes to code. These tools provide a running history of code development and help to resolve conflicts when merging.

The first source control tool I remember using was subversion. The windows client for this was called TortoiseSVN and is still available today. I know this as my current job has some legacy code which still uses it.

Back then, to use Subversion you needed to set up a linux server with subversion on it, and you would then connect over your LAN to it from your development machine. This was before the days of github, where everything was hosted on a SaaS service somewhere. I do not remember the process of getting everything setup, but with it being linux, there was no doubt a load of config files to edit, and different dependencies to install.

Over the years I installed various web interfaces to allow “browsing” of source code, some even had bug trackers built in. The ones I can remember are Trac and Redmine . Both look to still be available and you can probably install them if you want to remember way back when.

The first hosted source control tool I remember using was bitbucket and this as far as I can tell has only ever supported git. So I am guessing I imported my subversion repos into git ones at this point. The reason for bitbucket was chosen was it allowed private repositories for free, at that point github was probably available but only provided public repositories.

After bitbucket the git repos where moved to Azure DevOps or Visual Studio Team Services as it was called back then. This move was mainly to take advantage of the builds and releases feature and to decommission some aging build servers.

This brings me to today where I have a mix of public repos on github and private repos on Azure DevOps. For building my code I use a mix of github actions and Azure Pipelines. As Microsoft own both services now, there is a fair amount of crossover between the two services, however it is far from certain which service is the future.

Wow, my code has moved around a lot over the years. From no source control, to subversion, to git, to Bitbucket to Azure DevOps!

Visual Studio Keyboard Shortcuts

Tue, 05 Jul 2022 21:00:45 +0000

I use keyboard shortcuts all the time, mostly without thinking, but they can be quite a time saver especially for frequently used tasks.

I recently came across this list of Visual Studio ones. https://docs.microsoft.com/en-us/visualstudio/ide/default-keyboard-shortcuts-in-visual-studio?view=vs-2022

I was going to do a top ten, but not really sure how you rank keyboard shortcuts, so instead I will mention some of my favourites.

Build solution

If you are using Visual Studio you need to build your code. If you are anything like me, after even the smallest change I want to see what affect that has had so a rebuild is often needed.

Ctrl + Shift + B / Build Solution
F5 / Run
Ctrl + F5 / Run with no debugging

Cut, Copy, Paste, Undo, Redo

Programming is sometimes refered to as copying and pasting from StackOverflow , but they are always times you want to copy and paste stuff so the standard keyboard shortcuts are all here.

Ctrl + C / Copy
Ctrl + X / Cut
Ctrl + V / Paste
Ctrl + Z / Undo
Ctrl + Y / Redo (or Undo your Undo)

Comments in your code

Often we like to temporarily comment out a line of code to see what affect it has.

Ctrl + K, Ctrl + C / Comment selection
Ctrl + K, Ctrl + U /Uncomment selection

Using statements

I like to keep my using statements tidy, and not clutered up with ones that aren’t used anymore.

Ctrl + R, Ctrl + G / Remove and Sort Usings

Go to Implementation

I like to move quickly between methods that a class is calling to find out what is happening so the go to implementation and go to definition shortcuts are very useful. I tend to use the right click option more than the sortcut, but I would save myself a heap of time if I learnt these shortcuts.

F12 / Go to Definition
Alt + F12 / Peek Definition
Ctrl + F12 / Go to Implementation

Find file in Solution Explorer

You are editing a file and you have no idea what folder it is in the Solution Explorer. There is a shortcut to find out where you are.

Ctrl + [, S / Select Active File in Solution Explorer

Search

Finding references or a certain string is a common activity

Ctrl + F / Find in current file
Ctrl + H / Replace
Ctrl + Shift + F / Find in all files

Visual Studio tries to be helpful

Often Visual Studio will try and be helpful with a lightbulb or similar icon, instead of clicking you can press this shortcut to

Ctrl + . / Show Visual Studio suggestion

Learn some more

As I said before keyboard shortcuts are great time savers. Go check out the complete list and share with me the ones you found most useful.

Diagrams with Mermaid

Sun, 19 Jun 2022 18:00:45 +0000

I have been wanting to produce a diagram of the architecture of my side project for some time, but I have put it off as never sure what the correct tools is for this job.

Mermaid is a tool that lets you create diagrams from code and text. I first came across this tool about a year ago for use in a project. I had forgotten about this tool until the other day, when I was thinking about this problem again.

Is it possible to embed a mermaid diagram in the markdown used in a github repo? Well the answer is yes, so lets look at how that works.

A simple mermaid diagram looks like this:

graph TD;
    A[fff]--->B;
    A--->C;
    B--->D;
    C--->D;

which renders like this:

Lets look at what it is doing.

TD means the chart is top down.

A is the name of a node in the chart, [fff] is a label being applied to it.

Then we just define the relationships between the different nodes, you can have <—, —> or —, or even
<—>

Lets look at my architecture. I have a database (cosmosDB), I have a website running on Azure Static Web Apps, I have some Azure functions for getting data into and out of my database. I also have Application Insights monitoring the whole thing. I also have a console app for doing some data import stuff. This produces a diagram like this:

The code to produce this and display it on my github repo is simply

```mermaid
graph TD
    A[Azure Static Web App]---B[Http Fn]
    B---D[Database]
    C[Timer Fn]---D
    D---E[Import Console App]
    F[App Insights]---A
    F---B
    F---C

Pretty nice for a few lines of code to show what system talks with what.

There are improvements that can be made to the diagram, for example changing the shapes of the different services.

If you live in the AzureDevOps world you can add a mermaid diagram to your wiki pages (but I don’t think to your markdown files). Just use the following syntax

::: mermaid
sequenceDiagram
    Alice->>John: Hello John, how are you?
    John-->>Alice: Great!
    Alice-)John: See you later!
:::

For more info see the docs here

What diagrams are you going to build?

Scottish Summit 2022

Tue, 14 Jun 2022 18:00:45 +0000

On the 10th and 11th June 2022 Scottish Summit 2022 happened and I was there. It was an awesome event and I will attempt to summarise what happened.

Thursday 9th

On Thursday 9th I travelled down on the train full of excitement about the event. Once I got to the hotel, I was immediately greeted by James Cook , I was obviously in the right place, this weekend is going to be good.

Friday 10th

The hotel was less then 5 mins walk to the conference, which was being held at the Technology and Innovation Centre, University of Strathclyde. Registration was simple wave your tickets (which were free, apart from a contribution to charity), and you get your badge and a wrist band. (Pink for people contact, blue for hold back) I then got chatting to Keith Atherton , who amazingly lived in part of the world where I grew up. (Amazing what you learn at conferences!)

Time for the Keynote talk. Which of course because we are in scotland was introduced by bagpipes marching through the room! The talk was given by Connell McGinley and was titled Disability Awareness & Inclusivity. This talk was amazing. Connell is deaf and gave the entire talk with sign language and this was translated for the rest of us to hear. Connell talked about his career and the barriers he faced. Microsoft Teams was an amazing improvement to his work life, as the captioning allowed him to talk with customers for the first time!

The purple pound is the amount of spending power people with some kind of disability (15% of the worlds population) have and it is approx $13 trillion. If businesses can open their products up to more people it is a win for everyone. This was a really powerful way to open up the conference.

Next up was David Small talking about Azure DevOps pipelines and Power Platform. I don’t know much about Power Platform however all the DevOps stuff was great, some of it reinforced stuff I already knew, some was new.

Sam Cogan was next talking about Infrastructure as Code. Resources for this talk are on his github This was a great comparison of different tools for doing Infrastructure as Code. Terraform, Pulumi, Bicep, ARM templates where all covered. The difference between stateless and stateful tools. If you have a state file you need to look after it, so if you don’t need this think about using a stateless tool. If you are only using Azure consider Azure. If you are a C# (other languages are available) think about using Pulumi (which I have used and quite like)

Getting Started with Static Web Apps was next with Poornima Nayar this was a great overview of Static Web Apps but as I already use Static Web Apps this was mainly reinforcing what I already knew. However the Static Web App CLI was mentioned which I didn’t know about so I am going to spend some time looking at that! Another thing that was mentioned was GraphQL which is a language agnostic way of building APIs which I also want to learn more about. Resources for this talk can be found here

Next was Naming stuff in Azure with Barbara Forbes , she went through the challenges of getting naming right, especially with all the exceptions Azure throws into the mix. Tagging your resources can really help. Also Azure Policy can help report on what you have (and even enforce it, if you are careful). Lots of interesting stuff, as my naming strategy is non existent at the moment!

Defender for Cloud Apps with Charlie Gough . This was his first talk and he did really well despite technical problems. This was a security talk, I thought it was going to be about securing the apps you have built, but it was more about securing the apps that the users in your company are using. Still useful to know about, but not really my thing. Sorry Charlie!

Last talk of the day was ‘What I didn’t know about Office 365’ with Sharon Weaver Sharon went through Shifts, Planner and Forms all included in your Office 365 subscription.

Saturday 11th

To start the second day I had Maxim Salnikov talk about Static Web apps, if I had know it was about Static Web Apps again I may have chosen a different talk but this was still a great talk. Again the CLI was mentioned which I really need to investigate further.

Next up was James Cook’s talk Monitor Azure DevOps and GitHub with Sentinel. This was all about securing your code repositories with Sentinel and looking through the logs at what is happening. To use this tech you need either a GitHub Enterprise subscription or Azure DevOps using Azure AD Logins as it is designed more for the Enterprise than a one man band kind like me, however I will see if I can have a play around with it and see what I can learn.

Sarah Lean gave a talk about the state of IT Ops tools in the hybrid age. I enjoyed this talk, Sarah added extra fun elements like talking about the Scottish language and the highland cow. The first tool mentioned was Windows Admin Centre. I have heard about this but never used it, must have come out after I switched from Ops to Dev. Looks very Azure like. Azure Update Management was mentioned next, which I have used and works really well. This is the next generation of Update Management, a bit like WSUS from my server days. Azure Policy was mentioned again, a great tool for reporting and/or enforcing things. Azure Arc was mentioned, a way to bring Azure to you. With all these tools consider why you want to use them, what benefits do they bring, not just because it it shiny!

Mental Health with Rory Neary . This was one of my favourite talks. Rory talked about his mental health problems, his challenges, some of his coping mechanisms some of which involved the power platform. He encouraged us all to be more open about talking about Mental Health. Lots of talks have a solution to a problem, this one didn’t, but that Ok. We are all on a journey complete with its ups and downs.

Accessibility for Blazor with Dennie Declercq . I wasn’t as keen on this talk as some of the others, while I agree Accessibility is very important the solutions presented felt very generic and not really specific to Blazor. However I do have some ideas to try now and the accessibility browser extension sounds like a great idea.

After Hours

One of the best things about the conference was talking to people between sessions and afterwards. I am an introvert and tend to keep quiet but even I managed to talk to (for me) loads of people. Some of these I had previously connected to on twitter, and some were brand new on the day. I would encourage anyone reading this to reach out to the community, go to an event like this, you will be surprised how friendly and helpful people can be.

Scratch

Tue, 26 Apr 2022 18:00:45 +0000

My 6 year old boy has been playing with Scratch, so here is a quick introduction to what we have learnt about it.

Scratch is a website that allows you to “code” by combining blocks together.

This is how the scratch website describes its self

Scratch is the world’s largest coding community for children and a coding language with a simple visual interface that allows young people to create digital stories, games, and animations. Scratch is designed, developed, and moderated by the Scratch Foundation, a nonprofit organization. Scratch promotes computational thinking and problem solving skills; creative teaching and learning; self-expression and collaboration; and equity in computing. Scratch is always free and is available in more than 70 languages.

Scratch can be found by going to https://scratch.mit.edu/projects/editor/

A lot of the blocks affect your character or “sprite”, the default is a cute looking cat which may also be the scratch logo.

The left hand side of the screen consists of a list of the blocks of code you can use and your coding area where you can arrange them.

The right hand side shows the results of your code, where you can run and test what your code does.

It didn’t take my boy long to code something like this tonight.

This game moves the “sprite” towards the mouse pointer and a message “got you” appears on screen when it reaches your mouse pointer.

I won’t go through all the different blocks you can choose from, but I will highlight some of the programming concepts being learnt here.

inputs every computer program needs inputs and there are lots to choose from, things like moving the mouse or pressing a key on the keyboard.

outputs your code needs to do something, this code moves a sprite about the screen

variables my boy hasn’t heard of algebra yet, but he is already grasping concepts like variables, where a value can be assigned and maths can be applied to. In this example the score increases

loops this program keeps looping until it reaches the mouse pointer

logic other logic commands are also available, like if / else statements, also <, > and = statements can be used

There are lots of help and tutorials on the site to help you along, and you can customise the look and even record short sound clips to play in your code.

Temporal Trek Podcast

Fri, 18 Mar 2022 18:00:45 +0000

Earlier this year I was a guest on the Temporal Trek Podcast.

Your Inquiry is recognised.
Here is this weeks episode of The Temporal Trek.
“Dead Stop” part 2 of the unofficial Battle Damage two parter.

Special Guest @funkysi1701 who was paid with three containers of warp plasma. Bargain!https://t.co/up4zAKOpZy pic.twitter.com/0EUTAWmqgl
— The Temporal Trek Podcast(s) (@rider_coattail) March 18, 2022

Here is the episode I recorded: Temporal Trek Podcast Season 3 episode 30 Dead Stop

This week “Dan Dan” are joined by long time friend of the show, formerly from The Ten Forward Podcast, Simon Foster.

We discuss whether this is

A part two of a two-parter, last of a four parter or a previosuly unseen part 5 to the Tessek Prime Quintology?
Rogue full-AI or just a really super advanced computer?
Station seen again in Voyager?

Oh and by the way has anyone seen the guy who flies the ship, you know the one, tall bloke, loves a good prank, Space Boomer, answers to ‘Travis’?

It was great fun to record and I hope you enjoy giving it a listen.

AWS Cloud Practitioner

Thu, 17 Mar 2022 20:00:45 +0000

So today I sat the AWS Cloud Practitioner Certification Exam and passed! This is my second certification exam I have sat after the Azure Fundamentals one I sat last year.

This time was different than the Azure one. Colleagues at work encouraged me to look into doing this particular exam and my employer is going to cover the cost of the exam itself. Last time I did the exam in my own time and didn’t really discuss it with anyone from work.

The other difference was this time I went to a testing centre. The centre was a 5 minute walk from the office. So I decided to work from the office today and 20 minutes before the exam started I walked to the centre. I signed a couple of bits of paper, had my photo taken, had my passport and driving license laughed at, put my belongings in a locker and I was free to start the exam. No worrying about internet connectivity or kids making noise outside my room, or the endless photos at the prep stage of an exam from home. I think I prefer the exam centre experience.

The other difference was the first exam was Azure, this one was AWS. Both exams cover similar concepts, they are both introductions to Cloud Computing. It’s ten months since I did the Azure one, however they both had multiple choice questions and gave you the results at the end. I feel overly familir with Azure, while I don’t feel that was with AWS yet. A lot of questions relied on you knowing what AWS services are called. eg AWS Pipeline, AWS CodeStar, AWS CodeBuild, Amazon CodeGuru are all AWS services and are easily mixed up, especially as I have only used a few AWS services, while I have used a lot more of the Azure ones.

My memory is the Azure gave you a breakdown of your score at the end, while I am still waiting for that from todays exam. This difference may of course be due to it being in a testing centre.

I still prefer Azure to AWS, however today has proven that I can learn AWS stuff as well. For my next exam I want to get more hands on with my learning, as most of the services I have learnt about so far I haven’t seen in action.

Updating SQL Server with Docker

Wed, 23 Feb 2022 00:00:00 +0000

This morning I was listening to a podcast where the new features coming out for SQL Server 2022 were being discussed. This starting me thinking about what would be involved in upgrading.

Upgrading production environments is complex and there are licensing considerations to take into account. However for non production workloads like development this isn’t a problem so lets look at that first.

In the past I have installed SQL Server Devloper Edition onto my laptop, this is fine but I have found that unless you are very careful you may end up with multiple different versions of SQL Server sitting around, and it is difficult to cleanly remove them without a fresh install of the OS.

However in this day and age, Docker and Containers are king. My current development environment makes use of Docker and has a docker compose file which sets up SQL Server for this particular application, lets take a look.

  sqlserver:
    image: mcr.microsoft.com/mssql/server:2019-latest
    container_name: Sql
    ports:
      - "5432:1433"
    networks:
      - my-network
    volumes:
      - sqlvolume:/var/opt/mssql

This defines which docker image to use, in this case 2019-latest, sets up ports and the name and saves the data on a docker volume.

If we then run SELECT @@VERSION on this instance of SQL Server we get told:

Microsoft SQL Server 2019 (RTM-CU13) (KB5005679) - 15.0.4178.1 (X64)   Sep 23 2021 16:47:49   Copyright (C) 2019 Microsoft Corporation  Developer Edition (64-bit) on Linux (Ubuntu 20.04.3 LTS) <X64>

What if we change the docker-compose file to use 2022-latest?

manifest for mcr.microsoft.com/mssql/server:2022-latest not found: manifest unknown: manifest tagged by "2022-latest" is not found

SQL Server 2022 hasn’t been released yet so there is no docker image for it yet. Try this command again in a few months when it is available.

OK so what else can we try? What about a downgrade to 2017-latest? Will that work?

SQL Server 2017 starts but the following error gets logged.

2022-02-23 21:41:15.30 Server      Software Usage Metrics is disabled.

2022-02-23 21:41:15.30 spid6s      Starting up database 'master'.

2022-02-23 21:41:15.34 spid6s      Error: 948, Severity: 20, State: 1.

2022-02-23 21:41:15.34 spid6s      The database 'master' cannot be opened because it is version 904. This server supports version 869 and earlier. A downgrade path is not supported.

Doh we can’t downgrade the existing database we have. Probably a good thing really.

Microsoft release regular updates for SQL Server called CU’s (Cumulative Updates), you can see above we are on CU13. Is there a CU14 or CU15 we could try?

Update the docker compose to: mcr.microsoft.com/mssql/server:2019-CU14-ubuntu-20.04

At this point I actually got an error

2022-02-23 21:50:20.71 Server      Error: 17058, Severity: 16, State: 1.

2022-02-23 21:50:20.71 Server      initerrlog: Could not open error log file '/var/opt/mssql/log/errorlog'. Operating system error = 5(Access is denied.).

This is caused by trying to use SQL Server 2017 but it is easy to fix.

In docker desktop there is a volumes section, find the volume you are using with SQL Server and delete the errorlog mentioned above.

Now if you retry SQL will start OK.

Repeating the SELECT @@VERSION gives us a new CU

Microsoft SQL Server 2019 (RTM-CU14) (KB5007182) - 15.0.4188.2 (X64)   Nov  3 2021 19:19:51   Copyright (C) 2019 Microsoft Corporation  Developer Edition (64-bit) on Linux (Ubuntu 20.04.3 LTS) <X64>

Microsoft SQL Server 2019 (RTM-CU15) (KB5008996) - 15.0.4198.2 (X64)   Jan 12 2022 22:30:08   Copyright (C) 2019 Microsoft Corporation  Developer Edition (64-bit) on Linux (Ubuntu 20.04.3 LTS) <X64>

How much easier is this than manually installing updates and rebooting or attempting to uninstall and reinstall SQL Server. As SQL Server 2022 isn’t out yet I can’t say for certain what issues I may encounter but hopefully it will be as easier as this. And I don’t need to backup or restore and databases they are all available as before!

dotnet is 20 years old

Mon, 14 Feb 2022 20:00:45 +0000

Today Microsoft celebrated 20 years since the first version of dotnet was released with a special live stream event.

A lot has happened in the 20 years of dotnet (or .Net). It is my understanding that .Net was created to rival Java. When the .Net Framework was first created it was a windows only thing, but today modern .Net is a modern run anywhere platform, with .net applications running everywhere from PCs and Laptops, Raspberry Pi’s, Mobile phones and tablets (via Xamarin Forms), to Websites and Microservice APIs running on every Cloud platform out there.

If you want to celebrate this achievment tweet with the hashtag #DotNetLovesMe or download some of the digital swag available from github

On Thursday the 17th Feb the first preview of dotnet 7.0 is going to be released, with the latest version 6.0 only being released last November.

The above graphic comes from the following tweet.

This is 20 years of #dotnet releases. #dotNETLovesMe pic.twitter.com/Zxfe1SdWTq
— Khalid 🎟 (@buhakmeh) February 14, 2022

What do you like about dotnet? When did you first start using it? What are you going to build with it next?

Why do I have a website?

Tue, 25 Jan 2022 20:00:45 +0000

Is it to share my ideas? Is it to learn new technologies and techniques? Is it to create a following? Is it to educate others? Is it to build some kind of service? Is it some combination of all of these.

History

Back when the web was young and I was first learning HTML. I hand crafted web pages, adding photos I had taken with captions. If I needed a new page I just added a new html page and linked to it from another page.

As time went on I started to learn mysql and php and my website became a hand crafted php nightmare. I also applied what I learned to help my father run the website for his camera club.

At some point I started playing with WordPress. I have had various WordPress websites or blogs over the years. WordPress is very powerful you can do so many things, install so many plugins. WordPress runs on php and mysql and as my career started to centre around the .net space, I started to want something that was similar, so I could apply things I had learnt to my own website.

This has led me to the current state of my website. I have a WordPress blog, with most of my oldest content, my newer content lives on dev.to and I have a Blazor webassembly site that uses the dev.to api to run my new website.

Blazor webassembly is great, however it has some limitations which I am starting to push against. To host this as cheaply as possible I am using Azure static web apps, so no .net backend all the website is front end. I have some Azure Functions that does the backend bits that I need.

Google and other bots are not able to find any of my pages except index, due to the way Blazor works. I have got round this by pre rendering the content using https://prerender.io/

My next difficulty is how to generate a sitemap.xml or a rss feed for my blog. This has started to make me question my architecture decisions.

I could use a hosted solution like ghost which is popular with a some of my peers. This would solve many of the problems I am currently facing but I wouldn’t be able to play with everything as it is hosted and therefore someone else’s problem. How important that is I will look at later.

Another option would be to use github pages, there are quite a few ways to publish a github page, Jekyll and Hugo appear to be the most popular. Both produce static content and both are a new for me to learn. Interestingly I could also publish either to Azure Static Web apps if github pages ends up not being suitable.

Split in two

I think my website needs to be split in two. I need a stable blog platform probably using Hugo and github pages. This is what I want to get indexed by the search engines and be the primary way people find out about what I am doing.

I then have additional sites, that I use as my playground for learning new tech. I can easily link between them and I can tweak the style so they “fit” nicely together.

I am still considering what to do with dev.to. I like that I am using it as the backend for my blog posts, and its API gives me that flexibility to display that content where I want.

Using GitHub Actions

Mon, 10 Jan 2022 20:00:45 +0000

I’ve been running my website on Azure Static Web Apps for a while and it is pretty cool.

When you create a Static Web App on Azure you get asked for the github repo of your source code and even the branch to use.

Once you have selected this, you get asked for the type of code to deploy, mine is Blazor Web Assembly but you can use Angular, React or Vue.

You now have three variables to fill in the location in your code of the Website, the location of your Azure Functions and the output location usually wwwroot. Once you have set these three you can preview the GitHub Actions file that will be created and added to your repository.

I get something like this

name: Azure Static Web Apps CI/CD

on:
  push:
    branches:
      - feature/tempbranch
  pull_request:
    types: [opened, synchronize, reopened, closed]
    branches:
      - feature/tempbranch

jobs:
  build_and_deploy_job:
    if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.action != 'closed')
    runs-on: ubuntu-latest
    name: Build and Deploy Job
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: true
      - name: Build And Deploy
        id: builddeploy
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_<GENERATED_HOSTNAME> }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "Client" # App source code path
          api_location: "Api" # Api source code path - optional
          output_location: "wwwroot" # Built app content directory - optional
          ###### End of Repository/Build Configurations ######

  close_pull_request_job:
    if: github.event_name == 'pull_request' && github.event.action == 'closed'
    runs-on: ubuntu-latest
    name: Close Pull Request Job
    steps:
      - name: Close Pull Request
        id: closepullrequest
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_<GENERATED_HOSTNAME> }}
          action: "close"

This github action will run when you create a Pull Request to the branch mentioned in the file, or if you push code into the branch. This code get added into the .github/workflows/ folder and is the location that all github action workflows live.

I haven’t done much with github actions, however I have used Azure DevOps quite a bit. Over on the Azure DevOps side I have created a pipeline that deploys to a Dev environment, then a Test environment and finally a production environment.

Lets have a look at the workflow that I ended up with and with can break down how it all works. Note I am new to Github actions so if there is a better way of doing this do let me know.

name: Azure Static Web Apps

on:
  push:
    branches:
      - main
      - develop
      - feature/*

jobs:
  dev:
    runs-on: ubuntu-latest
    environment: 
      name: Dev
    name: Dev
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: true
      - name: Build And Deploy
        id: builddeploy
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_ORANGE_POND_09B18B903  }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "Blog" # App source code path
          api_location: "Blog.Func" # Api source code path - optional
          output_location: "wwwroot" # Built app content directory - optional
          ###### End of Repository/Build Configurations ######
  test:
    if: github.ref == 'refs/heads/develop'
    runs-on: ubuntu-latest
    environment: 
      name: Test
    name: Test
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: true
      - name: Build And Deploy
        id: builddeploy
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_WITTY_DUNE_0A1A77903  }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "Blog" # App source code path
          api_location: "Blog.Func" # Api source code path - optional
          output_location: "wwwroot" # Built app content directory - optional
          ###### End of Repository/Build Configurations ######
  prod:
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: 
      name: Prod
    name: Prod
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: true
      - name: Build And Deploy
        id: builddeploy
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_BRAVE_ROCK_0AAC63D03 }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "Blog" # App source code path
          api_location: "Blog.Func" # Api source code path - optional
          output_location: "wwwroot" # Built app content directory - optional
          ###### End of Repository/Build Configurations ######

The first thing I did was create three Azure Static Web Apps, I am using the free tier so while this is trippling my costs it is all still free! Doing this created three github action workflow files, I deleted two and edited the third, but before I deleted them I made a note of the AZURE_STATIC_WEB_APPS_API_TOKEN. If you look in your settings -> secrets for your repo you will see secrets have been created, this is the secure token that github uses to update your static web app.

While we are in settings we might as well look at environments. I created a Prod, Test and Dev environment that I was going to use in my github actions.

Environments can have various rules setup on them.

Required reviewers - this is like an approver, a user specified here must aprove for the workflow to be deployed
Wait time - I didn’t use this, but it looks like a certain amount of time can be set to pause the deployment. (I assume to do some kind of manual check)
Deployment Branch - specify what branch are allowed to be deployed to what environments. I specified develop, main and feature branches could be deployed to the Dev environment, develop and main could go on Test and main could go on Prod
Environment secrets - I didn’t use this as my secrets were already created, however it looks like your secrets can be associated with a specific environment

Now that we have the static web apps setup and the environments lets look at the github action file.

First of all I removed the PR stuff and just concentrated on pushes. I wanted my workflow to be.

Push to feature branch
Deploys to Dev env
PR feature branch to develop
Once merged code gets pushed into develop
Deploys to Test env
PR develop to main
Once merged code gets pushed into main
Deploys to Prod env (after approval)

The approval on deploying to production I think is probably overkill, but I still have it setup like that for now.

My gh action has three jobs defined as dev: test: and prod: they are all the same except they have the azure_static_web_apps_api_token that is correct for their environment.

They also each have a environment defined eg

environment:
  name: Prod

Lastly Test and Prod have an if test setup, if the test is false the job won’t run. Importantly it won’t fail it just won’t run.

For Prod this needs to only run on main branch so we have

if: github.ref == ‘refs/heads/main’

For Test this needs to only run on develop so

if: github.ref == ‘refs/heads/develop’

I could have a test for develop to only run on feature/* but I have allowed it to run everytime.

There is loads more you can do with github actions, but hopefully this gives you a taste of some of the things you can do. I currently have a mix of Azure DevOps and github actions so I will be working on getting github actions to do more.

2022 Goals

Sat, 01 Jan 2022 00:00:00 +0000

Lets have a look at what my goals were for 2021. I had eight of them, lets look at them one by one.

Azure certification. In May of this year I sat and passed the Azure Fundamentals exam. I am calling this goal as achieved.
Mentoring. I didn’t do anything about working with others or mentoring, so not sure I achieved this one. However I do now work in a development team and I have been reviewing others code and having my own reviewed. We have a junior developer and I am enjoying the opportunities I have to work with him and share my wisdom.
F#. I have done zero work with F# in 2021, so this one I didn’t achieve.
Cosmos db/Mongo db. I have worked with both of these technologies in 2021. I used Cosmos the most with my website, and storing data for it. I used Mongo/Atlas for auditing for a project I did for my previous job. I wouldn’t say I was expert in either of these, but I am starting to get a flavour of non SQL Server databases. I should also note that I am using mysql a lot in my latest job, so another non SQL Server technology that I get to use on a daily basis.
Give a talk. I said last year that maybe I would make baby steps towards doing this, and I have. In the interview for my new job I gave a short presentation (I thought it was bad, but others didn’t!) I also gave a short introduction to Blazor talk which went down well.
Mandlebrot Generator. Did nothing on this one as well. I may have googled the code but that is as far as I got.
Pwned Pass Mobile App. I got an increase in users in 2021 and hence it it still running and I still pay for the API key. I am still considering what to do with it.
Time for me. Achieved this one, had plenty of time for myself.

Not a bad year but what are my goals for 2022?

Video. I have just purchased a green screen, so my first goal is to learn how to use OBS, how to light myself properly without getting horrible reflections. I have the content for my first talk, the talk I gave at work about Blazor.
Conference. Attend an in person conference. I am booked to attend Scottish Summit, however it has been postponed due to COVID-19.
Blog More. I have been neglecting writing blog posts a bit recently, so I want to do more.
Metrics. I have been recording various metrics from twitter, github etc and I would like to expand this and make it a service.
Profile Pic. My profile pic is getting a bit old, so it would be nice to update the images I use online and improve my personal brand.

Back to BASIC

Tue, 28 Dec 2021 20:00:45 +0000

How exciting my Lynx Computer from my childhood has come home, all 96k of it pic.twitter.com/aeN38KBiS8
— Simon Foster (@funkysi1701) December 26, 2021

I can’t remember the syntax for BASIC, luckily I have been able to find the Manual .

All the commands are listed inside so lets see what we can do.

The Lynx presents you with a command prompt in which you can type text. Back in the 80s we had a tape player to load programs from tape, however I don’t have one today so only programs I write can be run.

PRINT - To write Hello World, you can just type PRINT “Hello World” and Hello World appears on the screen. To Write a program that displays Hello World, you just write the line number first.

10 PRINT "Hello World"

To run this you type RUN To view the code you type LIST

To Edit a specific Line you can use Ctrl+E and type the line number, or you can just write the line out again.

CLS - This command clears the screen

INPUT N - stores text typed by the user and stores it in the variable N

GOTO N - Execution of code continues at Line Number N

The first Program I wrote with a bit of help from my boys.

10 CLS
20 PRINT "What is your Age?"
30 INPUT N
40 IF N>5 AND N<41 THEN PRINT "a good age" 
50 ELSE IF N<6 THEN PRINT "a spaceman"
60 ELSE IF N>40 THEN PRINT "too old"

My 4yo didn’t like being “too young” in the original version, so my 6yo helped me change him to be a “spaceman”.

Not bad and it was fun pair programming with a 6yo, all my typos were quickly spotted, and he easily understood the logic of IF/ELSE/THEN statements.

The Lynx comes from 1983 and has just 96k of memory. I am very lucky it actually still works, however I have been able to find an emulator so I can write Lynx BASIC from the comfort of my laptop. jynxemulator , it is also on github but it doesn’t include the ROMs so getting from the website is a better option.

The developer experience today is so much nicer than it must have been in the 1980s, however back then distractions must have been much reduced.

No internet or google to get answers to your questions
No Copy/Paste of text
No Load/Save (unless you have a working disk drive or tape player!)
No IDE
No Build or Release process just type RUN

I then have additional sites, that I use as my playground for learning new tech. I can easily link between them and I can tweak the style so they “fit” nicely together.

I am still considering what to do with dev.to. I like that I am using it as the backend for my blog posts, and its API gives me that flexibility to display that content where I want.

A Comparison of All Good Things... and Star Trek Picard

Wed, 15 Dec 2021 20:00:45 +0000

The final episode of Star Trek: The Next Generation features a few scenes set 25 years into the future. That episode aired May 1994. The newest Star Trek TV show Star Trek Picard has just aired its first episode and this is 25 years and 8 months after that episode. This article may contain minor spoilers for the first episode of Star Trek Picard, you have been warned.

I thought it would be interesting to compare the two time periods.

Fate of Picards crew

In All Good Things… all of Picards crew are still alive except one. Deanna Troi passed away but it is never revealed how.

In Star Trek Picard, we know Data has died as featured in the feature file Star Trek Nemesis. From the trailers we know Riker and Troi are still alive. We do not know the fate of Crusher, LaForge and Worf yet, but I am going to assume they are still alive somewhere.

In both timelines we have one dead crew member.

Living arrangements

In both time lines Picard is living in France at his vineyard.

In All Good Things… we do not know who Picard is living with, it is assumed he lives alone, which is why Geordi feels the need to check on him, once he learns about his Irumodic Syndrome.

In Star Trek Picard we learn that Picard is living with a Romulan couple, due to the involvement Picard made to save the Romulan people when their star went supernova. Picard appears older but for the most part in good health, he can’t for example run up a flight of stairs (he is 92 so not surprising really!) and is plagued by dreams from his past.

Romulan Neutral Zone

In All Good Things… there is no Neutral Zone, the Romulan empire has been taken over by the Klingons.

In Star Trek Picard one would assume there is no Neutral Zone as well because the Romulan star system was destroyed. A handful of Romulans survived mostly due to the actions of Picard.

Looks

As we have seen Patrick Stewart does not appear to be ageing. Below is a comparison of how he looks in the two time frames.

I really enjoyed the first episode of Star Trek Picard and it is going to be interesting seeing what other similarities and differences there are between it and All Good Things…

Lone Developer to Senior Developer, my 2021 story

Sun, 05 Dec 2021 09:00:05 +0000

Welcome to Day 5 of the Festive Tech Calendar!

Usually at this time of year I like to take the opportunity to look back on the last 12 months, highlight some of my achievements and make a few goals for the new year.

Like many of you 2021 has been a difficult year, but it has also been a year of change, and I am in a much better place now than at the start of the year.

To tell you my 2021 story I first need to give you a bit of a history lesson about my career. So, lets fire up your flux capacitor (or Tardis of similar value) and journey back to 2006.

Back to 2006

In 2006 I got my first IT job. My housemate, who was working as an IT Manager for a small Health and Safety Company, suggested I come work with him. I was currently working an admin job with no idea what to do with my life, so I said yes.

So, in October 2006 I joined as a member of the IT Team. I worked mainly on first line support, but as the IT department was small, I learnt about all sorts of IT things, from fixing printers, setting up the CEOs BlackBerry mobile phone with emails, administering active directory, writing SQL queries to swapping tapes for the weekly backups.

The company had many faults which I won’t discuss here, but the work was varied, and I was always learning something new so I stayed with the company.

At the end of 2010, the IT Manager resigned, shortly followed by the rest of the IT department. In January 2011 I was IT Manager, I had no staff, plenty of IT problems and no clue what I was doing.

This was an amazing time for me, I learnt how to interview and hire staff, I learnt more about the different systems we used. It was approximately at this point in time that I switched from being exclusively a SysAdmin to starting to learn Development.

I had been creating simple websites for a while, mainly using PHP and MySQL. But since I started working in IT I learned more and more about databases, SQL Server and writing T-SQL scripts. I was the companies “database guy”, so the company naturally asked me to do more and more database work. This led to learning other technologies like MS Access and C# so I could do more and solve more of the company’s problems.

However, the company was small, and I would always be dragged in to fix SysAdmin problems so I never managed to spend 100% of my time doing development, so in 2016 I moved on to my first full time development job.

The new job was great, but I kept in touch with my old CEO, Ally and helped out with bits of work on the side, writing SQL queries, updating the odd bespoke application.

Back where it all began

In 2019 I started formulating a plan to go back to my old job as a contract developer. To my amazement Ally almost bit my arm off to get me back, agreeing to all my terms. So, in the summer of 2019 I started full time as a contract developer, mostly working from home with the occasionally meeting in the office and supporting the business with IT issues.

My second stint with the company was different than my first, I was able to concentrate almost exclusively on development work. As the only developer I was in full control of development, I decided to use dotnet and Blazor.

Late in 2019 I was given advanced warning that the company was being sold. In Feb 2020 the company did get sold, the new owners kept me on to finish the project I was working on but made me a permanent member of staff. In March 2020, the whole company began to work from home due to COVID-19, with parts of the business being furloughed.

This was a difficult time; on top of the stress the whole world was feeling from the global pandemic I was trying to assist with various IT integrations that you would expect when any company gets sold.

January 2021

This is pretty much where you find me in January 2021. I was working remotely developing internal applications with dotnet/Blazor with the occasional Teams meeting to demonstrate what I had built so far.

Working on your own as a developer is great! You start work and can write code all day. There is no daily stand up, you don’t need to explain what you have been working on, or what challenges you are facing. You don’t have any process or formality to follow, only what you impose on yourself.

I created a build/release pipelines in Azure DevOps, I created pull requests that would kick of a build, run my unit tests, and run some static code analysis. I would glance over my PRs but 99% of the time I would approve them.

This was the greatest weakness of working on my own, no one to suggest ideas of better ways of writing code, no one to bounce ideas off, no one to encourage or be encouraged when I figured out something clever or offer to help you overcome a problem.

Redundancy

In Mid July I was told that the development function was no longer required, and my job was at risk of redundancy. As I was the only one working in development it sounded very much like a done deal. I had a couple of redundancy meetings to attend and I would need to talk through how various applications worked as part of a handover.

To be honest I agreed with the decision to make me redundant. New development work was not coming through, although the users of the applications I had developed loved what I had created, upper management were keen to migrate off these systems and use more centrally managed systems. If I was in charge, I would probably have made a similar difficult decision.

As soon as I found out I was at risk of redundancy I called up a couple of recruitment agents I had spoken to before. They told me that it was a great time to be looking for a job and immediately arranged some interviews for the next week. I also took advantage of social media and asked on twitter if anyone was looking for developers like me. To my surprise this resulted in at least one interview.

One thing I stressed in all my conversations with recruiters and companies I spoke with, was that I was looking for a team. I wanted to work in a team, bounce ideas off others, mentor others, learn from others. This was the most important thing I wanted in whatever my next role was going to be.

Other important things for me was continuing to use Azure or similar cloud technologies. However, I always think that the tech stack and technology can be learnt on the job, tech moves so quickly these days that you have to be constantly learning to stay relevant.

About two weeks after I first found out about my impending redundancy I contacted (or more likely was contacted by) a third recruitment agency. It was this third one that eventually got me a job, however all three were brilliant, and kept me updated and answered all of my questions.

The next few days were packed with phone calls, interviews, tech tests. I had a lot of different types of companies that I spoke with, from Software Consultancy companies, FinTech companies, Energy suppliers, Legal companies and many others.

During this time of interviews, I can think of only two in person interviews. The rest made use of Teams, Zoom, Google Meet and even Skype (yes at least one company still uses it for video chats!)

The usual way the process went was a conversation with the recruiter about a role after which my details were sent to the company, If the company liked the sound of me an initial informal interview was arranged, after that a second more technical interview was held, sometimes there would be a tech test, sometimes it was more technical questions being asked in the interview. Some companies had more stages that this, but this was what I typically encountered.

I lost count of the actual number of interviews I had, but I used a spreadsheet to try and keep it all straight in my mind. The last thing I wanted was to ask a question about the wrong company!

Positive No

I got plenty of noes from these interviews, however all were phrased as a positive no. That being they liked me as a person but something about my skill or experience wasn’t quite right. A friend once described me as an odd mix of junior and senior developer. I agree with that statement, I have a lot of experience in some areas and a real lacking in others.

A common one was working in a team. Being a lone developer is not good for gaining experience working in a team. However, each time I spoke with companies I stressed that the team was what I was looking for.

Another weakness of mine is front end. When I am building something, functionality is more important than getting it looking good. I came from a database and backend start, so it is only natural that I am more at home working on these things. Also, my most recent projects are working with Blazor, which is brand new and few companies are doing much with it yet.

I tried really hard not to be discouraged by these noes and that the right company was just around the corner. One no hit harder than most. I had an initial chat with the team of a FinTech Company and then there was a second interview with the CTO which I felt had gone OK.

From what I had heard it ticked a lot of boxes, the team sounded good, with lots of support and development opportunities and they were moving to the Cloud, so my Azure experience sounded ideal. The CTO liked me and thought I would be a good fit, but I was part way between a junior and a senior, so he had gone away to try and make a bespoke role for me, eventually it would be a no.

Looking back, it is easy to see, that this was really encouraging. They tried to create a bespoke role for me! However, at the time all I could think about was all the noes I was hearing.

The job for me

About the same time, I had my initial interview with the company I would eventually accept a role with. I kept being told that there was a tech test to do, but for some reason it never got sent to me, so the interview featured a lot of tech questions and they really grilled me. I came away without much of a sense of if it went well or not.

However, they liked me and wanted me to do a presentation at their office. I am rubbish at public speaking and doing presentations, so I thought I will do this presentation, but it is very unlikely they will like what I do.

The topic of my presentation was how I upskilled a team on a new project or technology. As a lone developer what an earth could I do for a topic like that? I thought about what I had done to upskill myself on various things, however that doesn’t really address the question, as they wanted to know more about what I had done to upskill a team.

I thought back to a time when I helped get a largely un-version controlled codebase into source control and automated the build and release pipeline using Azure and Azure DevOps. This involved talking to SQL and data developers so could be a better option for a presentation.

I arrived at the office to do my presentation. First of all, I parked next to the factory where all the forklifts were, I was directed to the correct place to park. Then I tried to go to the wrong building, a very helpful employee helped me go to the correct place. I was introduced to the people who would be interviewing me, one of whom I had worked with in the past, who immediately said you should hire this guy, he is an excellent developer! As introductions go, that’s not bad!

In my opinion the presentation was bad. I didn’t have a PowerPoint or similar to go with my talk, so I just waffled on for ten minutes about what I had done a few years back about how I helped get source control being used.

The next day I got the job offer.

I was very surprised to get the offer as I thought I had done so badly the day before they would give me another positive no. I took a few hours to think about it, however all my other applications were either not started or waiting for the next stage, so I was 99% sure I would accept.

I had no idea what to expect when I started, as I didn’t really ask many questions at the last interview as I was convinced, I was going to get a no after my presentation.

I worked my notice period and at the start of October I started my new job. My contract only arrived the day before I started, so in the days before I started my mind was making up reasons why the job would disappear.

The job started with a two-day induction. Things that were covered on the induction were, mental health, sleep cycles, company values, health and safety and of course a bit of form filling and photocopying passports that you would expect on a first day.

Conclusion

Back when I had been job hunting, each time I stressed the importance of team. The company I had joined had an amazing focus on supporting its staff and getting the best from each other. I had got the best possible fit for the kind of team I was looking for.

Usually when I start a new job, I get a large dose of imposter syndrome. Now, two months after I started, I can say, I didn’t really get that this time. Maybe it is how the team functions, maybe it is my level of experience, or maybe its just something else.

I am happy in my new job. There is a lot to learn, both technology and how to work well in this team. But I am supported, there are people to ask question, I am contributing almost from day one. I have already shared a brief talk about my experience with Blazor, which went down better than I expected. I am excited to learn what is next for me.

If you have read all the way to here, Thank you! This is my story and I hope you have found it interesting. If you are looking for a new role, I would encourage you to focus on the soft skills you are looking for next, if you concentrate on team like I did, there is a good chance you will land in an amazing team.

Don’t forget to check out some of the other great content that is being created by all the amazing Festive Calendar 2021 contributors.

More than halfway through 2021

Tue, 31 Aug 2021 20:00:45 +0000

We are over halfway through 2021, let’s have a quick look at some things I have done this year:

Deployed a new version of the Pre Qualification questionnaire website for my employers
Passed Azure Fundamentals Exam
Mongo DB Atlas - experimented with using it for auditing my application
SonarCloud - tried to improve the quality of my code
Azure DevOps API -> built an application to show builds and releases from my Azure DevOps organisation, code is on github
Experimented with replacing SQL Agent Jobs with Azure Functions, ended up abandoning this project but was a great learning experience
Played with MS Graph - displaying profile pic from Azure AD but I am the only one in my org with a photo so abandoned this as well
Added a captch to a website to reduce spam
Charted my Gas and Electricity usage using the Octopus Energy API
Charted various metrics from services I use, eg twitter followers, github commits, blog posts published
Started learning react, easily connected it with Azure AD
Integrated my employers project management system with a postcode API and google maps so can see where in the country different projects are located
Tested dotnet 6 preview and the latest Visual Studio 2022 preview
I was made redundant and got a new job
Wrote my first piece of dotnet code to run on a Raspberry Pi
Kept cost of Azure down, by removing unused services, refactoring and optimizing existing services.

Wow, I have done loads this year, what is next?

My road to Certification

Tue, 11 May 2021 20:00:45 +0000

So today I sat the Azure Fundamentals Certification Exam and passed!

Really pleased with myself at achieving this. It was one of my goals for 2021 so I can tick that off.

Back in January I booked my first exam, however due to technical problems I didn’t get as far as the Exam. In order to be able to sit an exam like this you need a webcam and microphone so you can be monitored remotely to check that you are not cheating. Something in my network was blocking them from seeing my video so it got cancelled before it started.

My theory is that either my internet connection was playing up at the time, My Pi Hole was blocking something, or something else on my network was blocking the video feed.

This cancellation put me off Certifications, as I wasn’t sure how to debug the issue and find out exactly what the problem was.

A few months back I won a free Azure Certification for taking part in a skills challenge run by Gregor Suttie and Richard Hooper and this was due to expire at the end of this month so I thought why not book a second exam and see if I can get past the technical problems. I didn’t do any exam prep as I thought, I wouldn’t get that far.

This time I connected directly to my router, bypassing most of the network, firewall and other items on my network that could possibly cause an issue.

I cleared my desk, took photos of my ID, took pictures of my desk from every orientation, and waited for the technical issues to start.

I was in a queue waiting for the exam to start.

A connection issue has occurred you have been sent to the back of the queue. Oh here we go again!

But no I connected with the invigilator, who asked to confirm my monitors were disconnected and to move my wallet out of reach off my desk.

And we are off, I am answering questions from the exam.

I won’t go into detail about the questions, but I whizzed through them, most made me think, some I guessed at. I wasn’t expecting to pass, I thought maybe half marks or just under due to my familiarity with Azure. (I have been using it for years which must count for something!)

I was wrong I passed comfortably and now I have my first certification.

My advice for you if you have been using Azure for a while is to book this exam and see how you do, you may well pass like me. There are plenty of opportunities to get a free exam, attending conferences like Ignite often qualify you for one, look out for challenges and competitions by #AzureFamily people on twitter as they are very encouraging and helpful in your Certification journey.

And yes I am thinking about what Certificate to do next!

DotNet on a Raspberry Pi

Mon, 10 May 2021 20:00:45 +0000

I have had a Raspberry Pi for a few years and recently I connected it up again, I plugged in the camera and everything worked.

To start off you can view photos from the camera with the raspistill command. With a bit of clever scripting and the crontab I got the Pi taking pictures every 60 seconds. Even managed to take a nice picture of a robin.

However scripting isn’t really programming, and I would like to write a bit more code. Dotnet can run everywhere these days and it made sense to see if it would run on a Raspberry Pi.

@pete_codes has written a nice guide to getting started with dotnet on a Raspberry Pi https://www.petecodes.co.uk/install-and-use-microsoft-dot-net-5-with-the-raspberry-pi/ This guide and the nuget package https://www.nuget.org/packages/Unosquare.Raspberry.IO/ was all I needed to get started taking pictures with my Pi.

My initial goal is to take some wildlife pictures, stick my camera to a window and take pictures of what flies/crawls/jumps past the window.

The code I have written so far is available on github https://github.com/funkysi1701/RaspberryPiDotNet

So far the code takes a picture, uploads this file to Azure Blob Storage (so as not to fill up the Pi with too many image files) and deletes the image locally.

Run a dotnet publish -c Release and then cron can run dotnet RaspberryPiDotNet.dll (with full paths to the relevant files)

I then use crontab to execute the code every 60 seconds.

My code has an appsetting.json file which has a couple of settings that need completing for my code to work.

Storage: This is the connection string for Azure Blob Storage LocalPath: This is the path to where the camera will save its photos to, something like /home/pi/ is all you need but feel free to specify what you need.

Once the photos are in blob storage I plan to display them somewhere, add options to delete what I don’t want, maybe do something timelapsey.

I also don’t have a proper release pipeline and this grates on me a bit. I have been doing a mixture of writing code in VS and pushing that to github and then doing git pull on the Pi, and also writing code directly on the Pi. (VS Code can connect via SSH which is pretty cool!)

Charting my Energy usage with the Octopus Energy API

Sun, 07 Mar 2021 20:00:45 +0000

Back when I was a kid, I used to record our weekly gas and electricity meter readings in a little notebook. We then typed these reading into a spreadsheet (this was in the pre-Excel days), which allowed plotting as a line graph.

How would I go about doing a similar thing today?

First off, I have a smart meter that submits meter readings every 30 minutes or so. However, I do not know anyway to get access to these readings directly, short of manually recording them like I did 30 years ago!

Octopus Energy have a public API which allows you to pull your consumption readings. The smart meter sends your usage to your energy supplier, in my case Octopus, they then process these readings and allow them to be queried with an API they have created. It is not a direct connection to your data, but it is the next best thing.

Other energy suppliers will hopefully follow this example and allow users access to their consumption data.

How do I use the API?

Using the API is straight forward. Octopus supply you with a secret which you use to authenticate against the API with Basic Auth, no password just a username. Then you just need to pass some details of your meters to get an object containing the last few days meter readings.

API Docs

GET /v1/electricity-meter-points/{mpan}/meters/{serial_number}/consumption/
GET /v1/gas-meter-points/{mprn}/meters/{serial_number}/consumption/

{mpan}/{mprn} of your gas or electricity meter, and {serial_number} is the serial number of the meters.

Something to be aware of, I initially collected the last days consumption, which worked, however on one day I encountered a gap in the data for electricity. So, I changed to collect and store the last month’s data. I can then query this for what I need.

Usually, the last 24 hours of data is available after midnight of that day. e.g. at midnight 2nd March all the data for 1st March should be available. This is not guaranteed so don’t rely on it, however I see no problem with having a few days delay between charting your usage.

I am still testing this out but so far, I have three charts for gas (and the same for electricity), the first chart covers a 24-hour period, the next covers a day total over 2 weeks, the final chart covers a total for each month (as I write this I have less than a month’s worth of data!)

For the day and 2 weeks charts, I plot a comparison line of the previous period so you can easily compare the current and previous usage. From my limited testing I have already discovered my usage is very similar day to day.

Another point of interest is that gas consumption is in m^3 and electricity is in kW/h.

If you are interested in trying the Octopus Energy API, here is a referral link .

Azure DevOps Release Pipelines Pre and Post Approval

Sun, 14 Feb 2021 20:00:45 +0000

Azure DevOps release pipelines have lots of options to do things how you want. One of my favourites is the option for approval.

There are two ways you can do approvals Pre and Post deployment. Lets look at both.

Pre Deployment Approval

Lets imagine you have a simple deployment pipeline that deploys to a test/development environment before deploying to a production environment.

Pre Deployment Approval happens immediately before the release so in this example, click in the ellipse before the Prod release step.

You will get a screen like the above, you can select what users need to approve it and how long approval waits before timing out, the default is 30 days, but I tend to use a shorter time out of 3 days.

Post Deployment Approval

Post Deployment Approval happens immediately after the release so in this example, click in the circle after the Test release step.

You will get a screen like the above, with the same settings as before.

That is pretty much all there is to approvals so either option will prompt you to approve before anything gets deployed to your production environment.

Deployment Hours

To complicate matters I make use of the following setting to define deployment hours.

This setting will start the Prod deployment at 3am Mon-Fri.

If I configure Post Deployment Approval, as soon as my deploy to Test has completed a request for Approval is sent.

If I configure Pre Deployment Approval, at 3am Mon-Fri a request for Approval is sent (not ideal if you tend to be asleep at 3am)

So it looks like Post Deployment Approval is more useful for my use case. However if you deny approval either in Pre or Post approval this will mark the deployment as failed and show Red in your list of deployments.

From a casual glance it looks like the deployment to Test is failing, it isn’t I am just opting to not continue my deployment to production.

My Pipeline

This is how I have my pipeline setup. Deployment happens on Test and doesn’t have a post approval step.

After Test an empty stage called Approval runs and that has a post deployment approval, this happens immediately after Test so you get asked straight away for approval.

Prod does not start as I have my deployment hours configured. Once it is time for deployment to Prod to start it executes.

Now a casual look at my past releases, you can easily see which have been stopped by approval and which have failed due to whatever issue, and which have run all the way through to Prod.

And deployments to Prod can only ever run during my defined deployment window.

I am interested to hear how you have your deployment pipeline setup. Do you make use of Pre or Post Approvals? Do you ensure deployments always happen at specific times?

Goals for 2021

Mon, 28 Dec 2020 00:00:00 +0000

I didn’t make any goals for 2020, or if I did, I didn’t officially announce them. 2020 has been a hard year for all of us, 2021 is going to be better.

Here are a few ideas for my goals for the year ahead.

Azure certification
Mentoring
F#
Cosmos db/Mongo db
Give a talk
Mandlebrot Generator
Pwned Pass Mobile App
More time for me

I want to get a certificate to show how much I know. The obvious area for this is Azure. I spend a lot of time playing with Azure, building and deploying to the platform I should be able to get certified in this area. I actually had an exam booked in 2020 but it was an in-person exam so was cancelled when Covid hit. I plan to sit the foundation exam in the first quarter of 2021 and I will take it from there.
I have had a few times recently where I have been reminded that I don’t have much experience working with others. Mostly because I work in a one-man development team so there are limited opportunities in the workplace. I have a few ideas to change this, but this is one of my top priorities for 2021.
& 4) My experience is very Microsoft and C# so I want to spend some time exploring and learning tech that is adjacent to this. A functional language like F# sounds like a good compliment to my existing skills and my data skills are also very SQL server so some document database skills would be a good place to spend some time.
Doing a talk has been on my list for years. I am terrified of doing one and with everything online now I don’t know if that makes things easier or harder. I don’t think 2021 is going to be the year for this but maybe I will take baby steps towards this goal.
Twitter recently reminded me about watching a Mandlebrot generate one pixel at a time in the 1980s. I would love to explore the code used to generate them and how fast they currently are to produce.
My Google play store Xamarin forms app that uses the HIBP API has less than 100 users and I am considering closing it down, especially as I pay a monthly fee for API keys. I haven’t really decided what to do, maybe I will spend some time improving it, maybe I will close it down, maybe I will build something else.
For many people 2020 has been a hard year, I am one of those

Building a Twitter Clone

Tue, 22 Dec 2020 20:00:45 +0000

I saw a tweet about building a twitter clone being harder than you would think. So this of course started me thinking how I would go about building something like that.

Ok so where would I start? First a few assumptions.

Development by a lone developer ie me
Tech stack will be dotnet and other tech I am familiar with
Database backend, probably SQL Server but I might use table storage for cost reasons should I try and actually build this. However if I design this well this should be something that could be swapped out as the system grows
User accounts on the system will be small as I can’t imagine anyone ever signing up. Why sign up to a social media platform with no users?

I guess the next question is what is Twitter?

A website that allows you to share 280 characters of text with your followers, allows you to follow other users updates and allows other user to follow your updates.

It also has an API that allow you to do almost everything that you can with the website.

Then there are of course mobile apps to consider but I am going to assume this is out of scope, however assuming a good enough API then this shouldn’t be a problem for future development.

First Steps

To start off with I would concentrate on the API, and then build a web client that makes use of the API.

So what would my MVP (minimum viable product) be?

User can authenticate with my API to get a token which allows access to other API endpoints
User can create a tweet
User can view own tweets
User can view tweets of another user
User can view tweets in their timeline
User can follow/unfollow other users
User can search for other users
User can search for keywords in tweets

I think that is probably sufficient to build my MVP for the API.

An interesting side note is that I could use the OAuth Twitter authentication to allow users to login to my twitter clone with the real twitter login details. However this makes no sense to me as we are essentially adding a dependency on the real twitter.

So what would I use for the frontend? I would start off with a Client Side Blazor frontend. Once I had a proof of concept that worked, I would think about styling and adding the UI elements that are familiar to twitter users.

The beauty of Client Side Blazor is that I can host cheaply in azure storage and distribute around the world via a CDN.

Due to the high number of times that follower and following count and other stats are queried I would consider storing these in the database and include a regular job to recalculate them so they don’t get out of sync with the data.

Having said all of this I am very tempted to fire up Visual Studio and see how far I get, and what problems I encounter along the way.

#005: Twitter Cards

Sat, 12 Dec 2020 20:00:45 +0000

It has been a bit of a mad week this week. I joined a new team so lots of time learning what’s what and also being pulled in two directions as usual demands come through on top of that.

My blog runs on Blazor and I have been making use of JavaScript interop to update the html headers and update the page title to match the blog post article. This works great, I load the page and check the headers and they were saying what I wanted.

The problem was I wanted to add tags for twitter cards This means that when I paste a link to my blog on twitter you get a nice preview and pic of me in the tweet. This was not working at all even though I had the correct headers.

I eventually figured out that the problem was the fact I was using JavaScript to update my headers after the page had been initially loaded. Twitter was fetching my page before these headers got added and therefore couldn’t see the twitter card headers.

My solution was to use invalid html. Not ideal but it works. I added the required html tags in the body of my page using Blazor/C# instead of using JavaScript to add them into the header. Twitter appears to not be fussy in finding them in the wrong place.

Twitter provides a validator tool at Twitter Card Validator which my website now passes.

Not much else to say this week, apart from I am missing Visual Studio and C#, I have been mostly using VS Code on Linux and looking at php which isn’t as much fun as my usual day job.

Weekly Update #004

Sun, 06 Dec 2020 20:00:45 +0000

I use sp_send_dbmail to send results of sql queries by email to business users. Recently an issue was raised that data was being cut off after 255 characters. To fix this I added @query_no_truncate = 1, however this stopped the column headings from being included. No idea why you can’t have all the data and column headings but there you have it.

What I am doing now is running 2 queries, one to get the headings, and one to get the data. In theory you should be able to combine them with a Union however you then have datatype issues for non text columns so I gave up with that idea.

My results have 60 something columns (don’t ask its for a data import into a third party system!) so I am not typing them all out. I can shove query results into a temporary table and then execute to get a list of columns.

SELECT name 
FROM tempdb.sys.columns 
WHERE object_id = object_id('tempdb..#TempTable')

However I need my list to be horizontal so I can use as column headers. I can use dynamic SQL and a pivot to flip them round.

DECLARE @cols AS NVARCHAR(MAX), @query  AS NVARCHAR(MAX)

SELECT @cols = STUFF((SELECT ',' + QUOTENAME(name) 
    FROM tempdb.sys.columns 
    WHERE object_id = object_id('tempdb..#TempTable') 
    FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'),1,1,'')

SET @query = N'SELECT ' + @cols + N' FROM 
(
    SELECT name 
    FROM tempdb.sys.columns 
    WHERE object_id = object_id(''tempdb..#TempTable'')
) x
PIVOT 
(
    MAX(name)
    FOR name IN (' + @cols + N')
) y'

Weekly Update #003

Sat, 28 Nov 2020 00:00:00 +0000

Been a quiet week, so wasn’t expecting to have much to write on here, however a few things happened worth talking about.

My First PR

A comment was made to me to do something with the postcodes that are in the system I am developing. Find out what projects have postcodes near each other, and that way work can be grouped together and reduce potential mileage costs of staff that need to visit these projects.

A quick google search found https://postcodes.io/ which has an API that returns nearby postcodes. It also has a C# wrapper https://github.com/markembling/MarkEmbling.PostcodesIO

A comparison of what was being returned from the wrapper and what the API said should be returned revealed that the distance between postcodes wasn’t being returned.

As the code was on GitHub I could easily see how easy or difficult it might be to add the missing bit of information. It was easy! So, I forked the repo and made the change. I published the change to a private NuGet repo in my Azure Dev Ops account. That way I could try my revised package to check it did what I wanted.

I left a message on the GitHub project letting the owner know I had a potential fix for an issue. The project hadn’t been updated in over a year, so the owner may not be interested, or the project may have been abandoned.

I was in luck just 17 hours after I left a message the project owner said to create a Pull Request, which I did and shortly afterwards my code had been merged in and an updated version of the package existed in the public NuGet feed.

I have been thinking about contributing to open source for a while. However, I had not seen a project I wanted to contribute to, or a problem that I knew how to fix until now that is.

PHP

This week I had a call asking me if I knew PHP?

I did, over ten years ago, before I got my first IT job, I spent time learning PHP and MySQL. I created a blog, and I also created a website for my Dad’s camera club. The code I created back then was awful. No shared code, all the code was associated with the page, or sorts of bugs occurred and as time went by it became increasingly hard to update. The site was well liked but I eventually lost interest and moved on to learn other things.

This call led to me talking with the head of IT, and later a couple of the developers who have since granted me access to the codebase of a project.

I haven’t had time to spend a lot of time looking at the code so far, however this is nothing like the PHP I had built before.

The project makes use of the laravel framework and the first file I opened had methods and classes, so apart from the syntax you could think you were looking at C#.

Another thing that interested me was the project used docker containers, it has automated builds as well. Lots of modern programming ideas that I had some ideas about. I am looking forward to learning more about this project and how I might contribute to it.

Weekly Update #002

Sat, 21 Nov 2020 00:00:00 +0000

I know Active Directory is fussy about clocks being in sync however not sure how todays issue happened.

I run my docker compose file from Visual Studio and I get a weird error.

SecurityTokenNotYetValidException: IDX10222: Lifetime validation failed. The token is not yet valid. ValidFrom: 'System.DateTime', Current time: 'System.DateTime'.

I deleted my containers, open and close Visual Studio a few times, nothing helps. Eventually I think to find out what the time is on my container. It has yesterday’s date. What has happened here? Surely recreating containers would have caused them to have todays date? I reboot and everything is fine again.

Turns out that it is a know issue, see https://thorsten-hans.com/docker-on-windows-fix-time-synchronization-issue I am using WSL2 and I have now changed back to using Hyper-V and the issue hasn’t come back.

Earlier in the week I spotted my build step was failing.

  - task: NuGetToolInstaller@0

Swapping to the next version of the step is all I needed to do to fix it.

  - task: NuGetToolInstaller@1

My guess is that support was dropped for this earlier version or there is some other incompatability with .Net 5.

Weekly Update #001

Sat, 14 Nov 2020 00:00:00 +0000

One of my favourite podcasts is Troy Hunts weekly update. In it he discusses stuff that he has been working on, plus some personal stuff. I am going to attempt to do something similar. It will probably take me a few of these before we get a look and feel that works.

Monday

A week off work, mainly to use it up before year end, plus want to get a few jobs around the house done.

I did ask the following question on Twitter.

Hey #azurefamily and #dotnet developers how do I get more involved in mentoring?
— Simon Foster (@funkysi1701) November 9, 2020

As a one person dev team, my biggest weakness is working with others so any ideas of how to change that are great.

Tuesday

Dotnet 5 is out! The latest version of dotnet is released by Microsoft and to celebrate there is dotnetconf to listen to. Due to time zones and family commitments, I haven’t listened to an awful lot of it but I did see the keynote and loved the 3 Scott’s chat.

Wednesday

My youngest son was 3 today, due to Coronavirus we didn’t do much but we celebrated as a family, and he even had a zoom call.

Thursday

Blazor has a new feature Virtualize where a list of items can only load the ones on screen. I have been trying to get this to work on my blog, works great running locally but not working in production yet.

Think I know what might be happening. I use Cloudflare to do my SSL, as Custom SSL certs for the cheaper Azure Web Apps is not supported. Something in Cloudflare is caching or interfering.

https://zimmergren.net/solved-asp-net-core-blazor-web-sites-does-not-work-with-cloudflare-html-minification/

Turning off HTML minification fixed my issue!

One additional thing I added to my Blog is the /config page which details some of the config settings. I think this probably came from https://www.hanselman.com/blog/adding-a-git-commit-hash-and-azure-devops-build-number-and-build-id-to-an-aspnet-website but it was a while ago when I first did this on another project.

At the moment we have .net Version, Commit and Build links.

The .Net Version is obtained from

@System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription

A few other bits of info can be obtained from System.Runtime.InteropServices.RuntimeInformation which I have included on the page for fun. There are probably security concerns with exposing all this info publicly so something to bear in mind if you try this.

Build Info is passed to my code by a build step

- script: '(echo $(Build.BuildNumber) && echo $(Build.BuildId)) > .buildinfo.json'
  displayName: "Emit build number"
  workingDirectory: '$(Build.SourcesDirectory)/src/WebBlog'
  failOnStderr: true

This simply passed the build id and number which are stored as variabled and saves them in a text file.

I then have a class that reads them and constructs a link.

using Microsoft.Extensions.Hosting;
using System;
using System.IO;
using System.Linq;
using System.Reflection;

namespace WebBlog
{
    public class AppVersionInfo
    {
        private const string _buildFileName = ".buildinfo.json";
        private readonly string _buildFilePath;
        private string _buildNumber = string.Empty;
        private string _buildId = string.Empty;
        private string _gitHash = string.Empty;
        private string _gitShortHash = string.Empty;

        public AppVersionInfo(IHostEnvironment hostEnvironment)
        {
            _buildFilePath = Path.Combine(hostEnvironment.ContentRootPath, _buildFileName);
        }

        public string BuildNumber
        {
            get
            {
                if (string.IsNullOrEmpty(_buildNumber))
                {
                    if (File.Exists(_buildFilePath))
                    {
                        var fileContents = File.ReadLines(_buildFilePath).ToList();

                        if (fileContents.Count > 0)
                        {
                            _buildNumber = fileContents[0];
                        }
                        if (fileContents.Count > 1)
                        {
                            _buildId = fileContents[1];
                        }
                    }

                    if (string.IsNullOrEmpty(_buildNumber))
                    {
                        _buildNumber = DateTime.UtcNow.ToString("yyyyMMdd") + ".0";
                    }

                    if (string.IsNullOrEmpty(_buildId))
                    {
                        _buildId = "123456";
                    }
                }

                return _buildNumber;
            }
        }

        public string BuildId
        {
            get
            {
                if (string.IsNullOrEmpty(_buildId))
                {
                    var _ = BuildNumber;
                }

                return _buildId;
            }
        }

        public string GitHash
        {
            get
            {
                if (string.IsNullOrEmpty(_gitHash))
                {
                    var version = "1.0.0+LOCALBUILD";
                    var appAssembly = typeof(AppVersionInfo).Assembly;
                    var infoVerAttr = (AssemblyInformationalVersionAttribute)appAssembly
                        .GetCustomAttributes(typeof(AssemblyInformationalVersionAttribute)).FirstOrDefault();

                    if (infoVerAttr != null && infoVerAttr.InformationalVersion.Length > 6)
                    {
                        version = infoVerAttr.InformationalVersion;
                    }
                    _gitHash = version[(version.IndexOf('+') + 1)..];
                }

                return _gitHash;
            }
        }

        public string ShortGitHash
        {
            get
            {
                if (string.IsNullOrEmpty(_gitShortHash))
                {
                    _gitShortHash = GitHash.Substring(GitHash.Length - 6, 6);
                }
                return _gitShortHash;
            }
        }
    }
}

The BuildId and BuildNumber properties just fetch the details saved into the text file during the build. This can then be passed to build the build link.

<a href="https://dev.azure.com/{OrgName}/{RepoName}/_build/results?buildId=@appInfo.BuildId&view=results">
    @appInfo.BuildNumber
</a>

Finally, the GitHash properties need to fetch the hash and shorthash of the commit which is a bit more complex. This is achieved using the following line in your build.

- task: DotNetCoreCLI@2
  displayName: 'Publish'
  inputs:
    command: 'publish'
    publishWebProjects: true
    arguments: '--output $(Build.ArtifactStagingDirectory) /p:SourceRevisionId=$(Build.SourceVersion)'

/p:SourceRevisionId=$(Build.SourceVersion) add the revision hash to [assembly: AssemblyInformationalVersion] during the build which can then be extracted using the gitHash property above, before being passed into the commit link.

<a href="https://github.com/{OrgName}/{RepoName}/commit/@appInfo.GitHash">
    @appInfo.ShortGitHash
</a>

Security Headers

Sat, 26 Sep 2020 00:00:00 +0000

Have you wondered what info you are leaking via your response headers?, do you want some kind of guide about what headers to set or remove altogether?

Head on over to https://securityheaders.com/ This is a site created by security expert Scott Helme that rates a URL based on what response headers it can see.

I am pleased to say www.funkysi1701.com is now getting an A.

So how do you add/remove headers in dotnet core?

In my configure method in Startup.cs I have the following code block.

app.Use(
    next =>
    {
        return async context =>
        {
            context.Response.OnStarting(
                () =>
                {
                    context.Response.Headers.Add("Permissions-Policy", "microphone=()");     
                    context.Response.Headers.Remove("Server");
                    context.Response.Headers.Remove("X-Powered-By");
                    context.Response.Headers.Remove("X-AspNet-Version");
                    return Task.CompletedTask;
                });

             await next(context);
         };
     });

I have only included a few of the headers I am adding as the excellent https://securityheaders.com/ can tell you which headers you should add and what options you might want.

Back to Blogging

Fri, 25 Sep 2020 20:00:45 +0000

My last blog post was over six months ago.

Covid 19 has hit the world, and I will be honest I have found it a challenging time.

My Blog had gotten into a bit of a mess. It had become fragmented with different versions of the same thing; I will attempt to explain what has become of my blog.

The original WordPress site can currently be found at https://www.pwnedpass.com/ I would prefer it to be on a sub-domain of funkysi1701.com but for some reason I haven’t been able to get that to work, not sure if it is a limitation of my hosting package. I like WordPress, it is very flexible, easy to get blog posts out there. But I want to write content about development and having a site I can tinker with is important to me.

Most of my WordPress blogs have been imported into dev.to and a few extra have been written on this platform. I like dev.to it is a wonderful place to share content and it has one or two extra features I like.

dev.to has an integration with Stackbit/Netlify and this became https://dev.funkysi1701.com . I like having a personal site, but having the same content as dev.to. To add content to this site all I need to do is write it on dev.to and some magic will go on behind the scenes and new content will be published.

However, as a developer I don’t like magic, I want to understand what is going on a fiddle with all the settings and make it do what I want.

dev.to has an API, I can build a site in .Net Core and make API calls to fetch the content I want. I understand APIs, I understand .Net and can customise my site exactly how I want it, plus play about with a .net website. This is what https://www.funkysi1701.com is now.

So what have I built so far. I have a Server Side Blazor site running .Net 5. Why Server Side and not Client Side I hear you ask? Well only because I have more experience with Server Side and know how to quickly create a website with that technology, I may change it as time goes by, but we will see.

I have two pages a list of my blog posts and a page that displays the content. Both of these use the dev.to API. I lied, there is a third page I hacked together to do some page redirection from the WordPress URLs. This is something I will change as time goes on.

There are lots of improvements I want to do, there are probably also lots of broken images or links as well. Hopefully, this will result in a good platform to blog about as well as on.

Testing for expiring SSL Certificates

Tue, 03 Mar 2020 20:00:45 +0000

Let’s Encrypt is amazing, you can easily add SSL certificates to any website and automate the renewal process. I have talked before about how impressive it is.

Once you start adding SSL certificates to your production sites however you may want to check when they expire so you don’t get caught out. You can always open your site in your favourite browser and view the certificate information and expiry date.

However there is a way to automate this check.

[Fact]
public void IsSSLExpiring()
{               
  var handler = new HttpClientHandler
  {
    ServerCertificateCustomValidationCallback = CustomCallback
  };
  var client = new HttpClient(handler);

  HttpResponseMessage response = client.GetAsync("https://www.example.com").GetAwaiter().GetResult();
  Assert.True(response.IsSuccessStatusCode);
}

private bool CustomCallback(HttpRequestMessage arg1, X509Certificate2 arg2, X509Chain arg3, SslPolicyErrors arg4)
{
  var now = DateTime.UtcNow;
  var expire = arg2.NotAfter;
  var diff = (expire - now).TotalDays;

  Assert.InRange(diff, 30, 1000);
  return arg4 == SslPolicyErrors.None;
}

This code gets the SSL expiry date from https://www.example.com and will fail the xunit test if the expiry date is less than 30 days in the future. I then schedule my tests to run regularly on all my environments with a Let’s Encrypt Certificate and this gives me advanced warning if a SSL certificate is about to expire.

The Assert.InRange(diff, 30, 1000) line will fail the test if the expiry date is less than 30 days or greater than 1000, but as the default expiry for Let’s Encrypt certificates is three months it will never be greater than 1000 days even with a freshly installed certificate. These values can be tweaked to suit your use case, however 30 days is enough time for me to investigate what is happening.

To execute my tests I use a scheduled build in Azure DevOps, but anything that regularly can run your tests will do the job.

The code above is just a simple example to get your started for my purposes I have put all my URLs into config files and just pass these into my tests, so I don’t need a custom test for every different URL.

Picard is back!

Thu, 08 Aug 2019 20:00:45 +0000

It has just been announced at the annual Star Trek Las Vegas (#STLV ) convention that Patrick Stewart is going to return to his role as Jean-Luc Picard in a new TV Show.

Few other details have been announced other than the TV show will take place 20 years after the film Star Trek: Nemesis and will feature the character of Jean-Luc Picard.

Lets run though a few things I would like to see from this new show.

Low on Action

Star Trek: Discovery is a great show and has brought Star Trek back to our TV shows. However it is an action show, it zooms along so fast and with so much action I would appreciate a show that can takes its time a bit more.

I want a show that can show Picard carefully considering some moral issue and making a decision. Something we were used to seeing on TNG.

Feature Picard’s excellent speeches

One of the best features I can think of Jean-Luc Picard is his ability to make speeches about one issue or other.

Lets look at a few examples:

Insurection : Picard speaks about the forced relocation of a group of people.

How many people does it take, Admiral, before it becomes wrong? Hmm? A thousand, fifty thousand, a million? How many people does it take, Admiral?

First Contact : Picard is hell bent on revenge for what the Borg have done to him and refuses to destroy the Enterprise to save his crew.

“I will not sacrifice the Enterprise. We’ve made too many compromises already. Too many retreats. They invade our space and we fall back. They assimilate entire worlds, and we fall back. Not again! The line must be drawn here, …this far, no further! And I will make them pay for what they’ve done.”

Measure of a Man : Picard argues for the rights of Data.

“You see, he’s met two of your three criteria for sentience, so what if he meets the third. Consciousness in even the smallest degree. What is he then? I don’t know. Do you? (to Riker) Do you? (to Phillipa) Do you? Well, that’s the question you have to answer. Your Honour, the courtroom is a crucible. In it we burn away irrelevancies until we are left with a pure product, the truth for all time. Now, sooner or later, this man or others like him will succeed in replicating Commander Data. And the decision you reach here today will determine how we will regard this creation of our genius. It will reveal the kind of a people we are, what he is destined to be. It will reach far beyond this courtroom and this one android. It could significantly redefine the boundaries of personal liberty and freedom, expanding them for some, savagely curtailing them for others. Are you prepared to condemn him and all who come after him to servitude and slavery? Your Honour, Starfleet was founded to seek out new life. Well, there it sits. Waiting. You wanted a chance to make law. Well, here it is. Make a good one.”

Ask important questions

We live in uncertain times. There are lots of pressures on today’s world and we need to acknowledge these. Picard is an excellent character to use to ask important questions of the day.

What does it mean to be human?
How should we treat people that are different to us?
How should we react to extremism?

Deal with his age

In the last episode of TNG a future version of Picard is shown 25 years into the future. This is a similar time period so it will be interesting to compare the two. Have Picard look back on his life and ask questions about his life choices.

Be consistent with his past

Picard is a well known character. So writers please don’t change him. I want a character we know, not a character that looks like Picard but acts in a way contrary to what we would expect.

Picard has a history we know, he served on the stargazer and of course the Enterprise, he has an artificial heart, he was assimiliated by the Borg. All

Reference 50 years of Star Trek

Discovery and Enterprise are TV shows which have a problem. There is Trek going on after them which imposes a few rules on what can and can’t be done.

This is a TV Show that will take place after every other Trek TV and Film and can do whatever the creators want. This gives freedom, but it also gives the responsibility to reference what has gone before.

The 2009 Star Trek film features the destruction of Romulus and Spock travelling back in time. How about Picard dealing with surviving Romulans helping rebuild their society, continuing the work that Spock started in the TNG episode Unification.

I look forward to finding out what the Star Trek production team has in store for us. If they have Patrick Stewart convinced I am sure it is going to be good.

Infrastructure as Code - Creating Azure Resources with Terraform

Sat, 11 May 2019 20:00:45 +0000

Terraform is a way to script the creation of Azure Resources. By scripting the creation of your resources you can run them again and again and be certain that everything is configured exactly the same as the script has specified. No more forgetting to tick the tick box, if it’s in your script then it will be deployed with exactly the same configuration.

The process of executing a terraform script follows this basic process.

terraform init – Initialises the terraform directory terraform plan – generates and displays what will be executed terraform apply – applies the terraform script to your Azure subscription terraform destroy – deletes your Azure resource that were created in the previous step

Lets look at how you might execute a terraform script from your build or release pipeline.

I am executing my terraform scripts from my release pipeline and as I don’t want to check binaries into my source code I am going to add a bit of plumbing to get things in the right places. The terraform binary can be obtained from https://www.terraform.io/downloads.html .

To start with I am executing a Powershell script to download the terraform binary, this is in the form of a zip file, so i need to download and unzip it.

#Download
$url = "https://releases.hashicorp.com/terraform/0.11.13/terraform_0.11.13_windows_amd64.zip"
$output = "terraform.zip"
(New-Object System.Net.WebClient).DownloadFile($url, $output)

#Unzip
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Compression.ZipFile]::ExtractToDirectory($output, Get-Location)

Now that I have the terraform binaries I can just run Azure Cli and execute the terraform commands above.

Now it is time to look at the the terraform script. This particular script creates a resource group, an app service plan and a webapp.

# Configure the Microsoft Azure Provider
provider "azurerm" {
  subscription_id = ""
  client_id       = ""
  client_secret   = ""
  tenant_id       = ""
}

resource "azurerm_resource_group" "rg" {
        name = "resource-group"
        location = "northeurope"
}

resource "azurerm_app_service_plan" "plan" {
  name                = "planname"
  location            = "${azurerm_resource_group.rg.location}"
  resource_group_name = "${azurerm_resource_group.rg.name}"

  sku {
    tier = "Standard"
    size = "S1"
  }
}

resource "azurerm_app_service" "webapp" {
  name                = "example-app-service"
  location            = "${azurerm_resource_group.rg.location}"
  resource_group_name = "${azurerm_resource_group.rg.name}"
  app_service_plan_id = "${azurerm_app_service_plan.plan.id}"
}

There are four guids that you need to fill in from your azure subscription and to find these you need to run a few commands from Azure Cli. Check out https://docs.microsoft.com/en-us/azure/virtual-machines/linux/terraform-install-configure for more detail on these commands.

Run these either from Azure Cloud Shell or from Powershell command prompt after Azure Cli has been installed.

az login – to login to azure a browser window will open up.

az account show –query “{subscriptionId:id, tenantId:tenantId}” This will show you your subscription id.

az account set –subscription="${SUBSCRIPTION_ID}" Where ${SUBSCRIPTION_ID} is the subscription id you found in the last step.

az ad sp create-for-rbac –role=“Contributor” –scopes="/subscriptions/${SUBSCRIPTION_ID}" Where ${SUBSCRIPTION_ID} is the subscription id you found in the last step and assumes you have permission to the subscription.

This last command will about a series of Guids which you need to fill in at the top of your terraform file appId = client_id, password = client_secret, tenant = tenant_id and subscription_id you already have.

This just gives you a taste of what you can do with terraform. Looking at the Azure Terraform docs there are loads of different resources you can create and even a github repo with some samples.

Gated Release

Fri, 05 Apr 2019 20:00:45 +0000

Automated releases of software are great but how can we add an element of feedback so only good releases go live.

I have been using Azure DevOps to release my PwnedPass android app to the Google Play Store for a while now. There are options to deploy to the alpha, Beta or Production tracks and even to set % of users to target. For the full range of options check out the Google Play extension for Azure DevOps.

My release starts by publishing to 10% of users on the production track, my next step makes use of the increase rollout option to increase this %, you can have as many of these additional steps as you want until you reach 100% of your users.

Now if you run this release now it will just run through each of the steps one after the other. Now of course you can add a pre or post approval to your pipeline but this just adds a manual dependency to your release. Whoever does the approving needs to check things are working before approving or worse just approves regardless.

Azure DevOps has the concept of gated releases which allows you to add automated checks before or after a release happens. These automated checks can be any of the following:

An Azure Function
A Rest API call
Azure Monitor Alert
Query Work Items
Security and Compliance Assessment

We are going to make use of the Azure Monitor Alert, to create an alert from your Application Insights data and only continue the rollout if no failures are detected.

Open up your application insights resource in the Azure portal and look in alerts. Click add new alert rule.

Select your application insights resource in Resource, In Condition choose a condition to check, I chose Failed Requests, so every time a failure is registered in my API I can stop the deployment. The exact criteria you want to use is entirely up to you.

Create an action group, I just set my alert to send an email to myself but there are other alert actions you may want to try. Give your alert a name and description and click save.

Now all we need to do is make Azure DevOps make use of this alert. In your release pipeline select the pre-deployment conditions of your second step and open up the Gates section.

Choose a suitable time to evaluate, I have been using something long like 12 or 24 hours so if there are problems there is time for it to be noticed. Choose Version 1 of the task (I was not able to get it to work with Version 0)

Now select your Azure subscription and Resource Group and leave the rest of the settings as they are. Now your Deployment will stop and analyse application insights for any Failed requests and will halt if it finds any.

I am still testing this out but it will take a few days to figure out if this what I want due to the large time scales involved. I feel this is going to be an improvement of manually approving release steps.

Documenting your API

Wed, 27 Mar 2019 20:00:45 +0000

So you have created a super API that does something amazing. How do you document it so people will use it?

One way of easily documenting your API is to install the Swashbuckle package.

Install-Package Swashbuckle.AspNetCore
Install-Package Swashbuckle.AspNetCore.Swagger

Then in you startup.cs add the following lines

//In ConfigureServices

services.AddSwaggerGen(c =>
{
    c.SwaggerDoc("v1", new Info { Title = "API", Version = "v1", Description = "An API Description" });
    c.IncludeXmlComments(string.Format(@"{0}\API.xml", System.AppDomain.CurrentDomain.BaseDirectory));
});

//In Configure

app.UseSwagger();

app.UseSwaggerUI(c =>
{
        c.SwaggerEndpoint("/swagger/v1/swagger.json", "API V1");
        c.RoutePrefix = string.Empty;
});

Now when you browse to your API you will see the swagger documentation system.

The RoutePrefix setting controls the path in which swagger will display. I have my docs at the root, but you might want them under the /docs or similar path.

The IncludeXmlComments setting from the ConfigureServices method allows you to load in any XML comments you have added to methods. For this to work you need to enable a setting to your build.

The XML documentation file must be ticked and contain a path. Everytime you do a build, a XML file will be generated which contains all the comment blocks you have added to your code.

Swagger will then use this XML documentation file to produce lovely looking documentation without you having to do anything extra.

Azure Key Vault

Tue, 19 Mar 2019 20:00:45 +0000

Azure Key Vault is a secure way of storing your keys, certificates and secrets so your application can access everything it needs to but you don’t have them being stored insecurely anywhere such as in source control.

I have been wanting to give Azure Key Vault a try for a while now as it can make use of Azure Active Directory to give your web app an identity so it can authenticate itself into the key vault to access secrets. Pretty clever but with a lot of moving parts a bit complex.

For my example I am just going to connect to my Key Vault and get a secret and display it somewhere on a web page. This is of course not what you want to do as secrets are secret and shouldn’t be displayed just used to authenticate into whatever, however it is an easy way to prove I am connecting to the Key Vault and everything is working.

Lets look at some code. I have a .net core application and to start with lets install three nuget packages.

Microsoft.Azure.KeyVault
Microsoft.Azure.Services.AppAuthentication
Microsoft.Extensions.Configuration.AzureKeyVault

I’ve not include version numbers as these will no doubt get updated over time but hopefully it will still work.

Now in your Program.cs add the following code, replacing [KeyVaultName] with the name of your Key Vault.

    public class Program
    {
        public static void Main(string[] args)
        {
            CreateWebHostBuilder(args).Build().Run();
        }

        public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .ConfigureAppConfiguration((context, config) =>
            {
                    var builtConfig = config.Build();

                    var azureServiceTokenProvider = new AzureServiceTokenProvider();
                    var keyVaultClient = new KeyVaultClient(
                        new KeyVaultClient.AuthenticationCallback(
                            azureServiceTokenProvider.KeyVaultTokenCallback));

                    config.AddAzureKeyVault(
                        $"https://[KeyVaultName].vault.azure.net/",
                        keyVaultClient,
                        new DefaultKeyVaultSecretManager());
            })
            .UseApplicationInsights()
            .UseStartup<Startup>();
    }

Now all you need to do is look at your configuration to pull out secrets from your Azure Key Vault. If you have a secret called AppSecret then you can use the following code snippet to retrieve its value, assuming _configuration is an implementation of Microsoft.Extensions.Configuration.IConfiguration.

_configuration["AppSecret"];

Now if you do all of this and run from an Azure Web App or run locally it will fail to pull anything from the Key Vault. You need to give your web app an identity and configure your key vault to allow access from that identity.

Once my code has been deployed to an Azure Web App I get the following error.

Lets look at fixing that, first lets give my web app an Identity. Open up the Azure portal and find the identity section of your web app and turn the setting on.

Now you need to grant that identity permission to your key vault. In the portal open up Access Policies in your key vault and click add Policy, select the identity of your web app in the principal box and select the following settings to grant access to your secret.

Now you have a website that can pull secrets out of Key Vault but only that unique identity. Anyone who has access to your source code will not have access to your secrets, even if they push your code to a different Azure Web App.

Technology I want to learn more about

Tue, 05 Mar 2019 20:00:45 +0000

While at Microsoft Ignite I heard about a lot of cool tech that I want to know more about. The best way to learn something is use it to solve a problem.

So what can I build that is both useful and will let me play with some new tech?

I have a Xamarin Forms app Pwned Pass that has over 500 downloads on Google play and over 80 downloads on the Microsoft Store . This has given me a small user base that I can use to make use of whatever I build.

My app makes use of the HIBP API created by Troy Hunt. I am going to build my own API, initially it will just make calls to the HIBP API. Building this will give me experience of building something with .net Core from design to deployment. I have made a start already on doing this, I have an empty .net core API project which deploys to an Azure web app using the build and release pipelines from Azure DevOps.

You may be wondering why I am not making use of Azure Functions to build this API. Azure functions is certainly a great technology that is worth learning about. However I have done a little bit with them in the past and I don’t believe I would be able to learn all the things I want to if I used Azure Functions. My primary goal is learning and sharing that learning via this blog. It may well be I move to using Azure Functions later on.

Another tech I am keen to learn more about is Azure Key Vault . This is a technology that allows the securing of keys, connection strings and certificates. I want my app to securely get keys and security information without any of it having to be committed to source code or shared insecurely.

Monitoring my app is also a key learning from me. I use application insights already, but I would like to extend my understanding of this so telemetry can be fed back into the build and bad deployments stopped.

Below is my complete list of learning and tech I want to touch on. It is a long list and I imagine it will get longer as I work through it. I want to regularly blog and share what I have been working on. I currently have a working build and release pipeline but nothing of note to build or release. I know Key Vault needs looking at early as the identity of the website in Azure is key to getting that tech working correctly.

Build API with .net core
Add build and release pipeline
Make use of Azure KeyVault for secrets, connection strings etc
Plugin My Xamarin app to make use of it
Monitor my API with Application Insights
Secure it with CSPs and log this into Report URI
Consider building a web frontend to my API using a javascript library or framework. Maybe react but this can be decided later.
Dockerize the API and add the creation of docker images to the build/release pipeline

Microsoft Ignite | The Tour – London

Tue, 26 Feb 2019 20:00:45 +0000

I have just spent the first day at the conference Microsoft Ignite | The Tour. The conference was free I just needed to arrange travel and accommodation. Microsoft have really looked after me and all the other guests, breakfast and lunch has been provided, I got a free t-shirt, coffee (or tea) all day plus a beer or glass of wine to end the day. And that’s before you collect any free stuff the vendors are giving away.

The first session was Designing resilient cloud applications with Matt Soucoup and talked about some cloud technologies like Azure key vault and serving static files from blob storage. Unfortunately this had a few technical issues with the demos. I think it was just connectivity with the MongoDB backend but this slightly spoiled the session. As this was the only technical problem I noticed all day I can let it pass.

Next was a session on Azure DevOps mainly build and release pipelines called Deploying your application faster and safer with Brian Benz . A lot of this I knew but good to reinforce I am doing things correctly.

Next was a session on Application Insights called Detecting application anomalies with Telemetry with Matt Soucoup.

Probably the most useful session was on Docker and Kubernetes called Integrate containers and Kubernetes into your Azure DevOps build and release model with Marco De Sanctis . Going to spend some time looking through the examples from this session.

A session on Serverless covered Azure Functions, Azure Logic apps and the other Azure Serverless offerings. Investing in Serverless: less servers, more code with Simona Cotin .

Lastly was a panel discussion on the changes facing IT Pros and SysAdmins. What is the future of the IT Pro in a DevOps & Cloudy world? with Jennifer Stirrup , Baki Onur Okutucu , Amy Boyd and Stephen Thair . Should they learn to code, how should they make sure they keep up.

Tomorrow I have a loads more sessions, including ones about mental health, Azure pipelines, more Kubernetes stuff and dealing with failure.

Yaml Builds on Azure DevOps

Thu, 31 Jan 2019 20:00:45 +0000

I have been using Azure DevOps (Or VSTS or VSO etc) for a while now and one of the great features is doing automatic builds with every check-in. This is more commonly known as a CI (continuous integration) build.

More recently I have started playing about with creating my build using YAML files instead of using the web user interface to create my build.

Why?

You may wonder why go to the effort of learning the YAML syntax when you can just create the build in Azure DevOps and then forget about it.

Mostly it is because the build changes over time and you shouldn’t just forget about it. If something changes over time then you might want to version control it, or look at a previous version.

Lets say you create a pull request that replaces a .net 4.7 web service with a .net core web service. If you have a CI build this PR will fail because it won’t build. If you change the build first any other builds going on will fail. What you want in this case is the build to be associated with that branch or PR. Any builds before you merge this change in will continue to work and any after this change will also work.

How?

How do you get started with YAML builds? Well the first thing is to make sure that YAML builds are turned on, as I write this I believe they are still a feature you can turn on or off. Have a look in Preview features and make sure they are turned on.

Next look at any of your existing builds and click the View YAML link. This will show you an example YAML file of your existing build. You could just save this as azure-pipelines.yml and checkin to the root of your project. You can also click the add new build pipeline option, this will give you some templates to start you off.

The YAML file consists of a series of build steps usually called tasks, with a few settings before to configure things like parameters or build agents. Detailed docs about the syntax of the file can be found here .

For my mobile app my YAML files consist of downloading nuget packages, building the solution, building specific projects with desired settings, running powershell or other scripts to set things up and finally publishing the results of the build as artifacts so that they can be used in any releases.

Secure It!

Avoid committing passwords and secure keys into source control. I have found you can upload secure files via Azure DevOps and then add a download secure files step at the start of your build. This allows the secure file to be used during the build but the contents of the file can’t be viewed by anyone with access to the source code.

I find it often takes a bit of thinking about how to achieve this, but it is usually possible to keep keys and secrets secure.

Pwned Pass Update

Wed, 23 Jan 2019 20:00:45 +0000

Its been a while since I first released Pwned Pass so lets have a look at where we are now.

We are very close to 500 Downloads from Google Play and we have recently smashed past 100 active installs, peaking at 116 and even now we are still over 100. I have had 9 reviews (6 x 5*, 2 x 1 * and a 4 *) which averages out at 4 * Over Christmas I released a UWP version that can be found in the Microsoft Store . This has currently had 9 downloads and even had a download to windows mobile (someone out there still likes the platform!) I have a fairly smooth deployment process using Azure DevOps. After every check in of code a build runs which compiles the UWP and Android versions. The build also increments the version numbers that is required to deploy to either of the app stores.

Every successful build of the master branch will kick off a release to the Beta track of Google Play. If I am happy I then release to 10% of the Production track, which can then be increased to 100% (or halted). The release to Microsoft Store happens after the Beta track of Google Play. Only reason for this order is that there isn’t a beta area for UWP apps so I want to quickly test change on android before rolling out for windows.

All these steps require confirmation by me before proceeding and often don’t get further than the beta track.

A further development is that I have open sourced the source code to github do take a look if you are curious or want to contribute. With the purchase by Microsoft there are easy ways to connect github repositories to Azure DevOps. Once I create a Pull Request in github it creates a build in Azure DevOps and all the build and release steps can happen.

I am still not 100% sure if I want to keep my bug and issue tracking in github or Azure DevOps as both have features for doing so.

One future improvement I want to make is to automate the creation of screenshots. When I create a new feature and it gets checked in. I would like to automatically created screenshots of the key pages and submit them to the different app stores. Currently I am not sure if this is possible or how to go about it. I have some ideas to experiment with so we will see what I can do.

Lets see what 2019 can do!

Tue, 01 Jan 2019 20:00:45 +0000

This is my annual lets make some goals for the new year blog post. So in no particular order.

Improve House

I can’t outdo last years goal of buying a house but I can continue to make efforts to improve it. I was going to put something about decorating or doing something to our bedrooms, however today I spotted a hole in the roof so this may well absorb most of my home improvement budget.

Connect with local area

Since we moved in October we haven’t really connected with our local area yet. My only interaction with my neighbours was when my car blocked his drive!

I am going to make efforts to change this in 2019. I don’t exactly know how yet, but I have some ideas. My son starts nursery and later school this year so could be one way to meet other parents and teachers. I want to connect with a local church, so far this hasn’t happened so need to make more effort with this. There are other local groups that put on child friendly events which would be good to attend.

Lightning Talk

I failed last year to do this so it is back on my list. I know I must have stuff of value to share so going to try again on this.

A lightning talk is a very short talk or presentation given at a conference or user group. I am not a natural public speaker so this is a step out of my comfort zone, however I am going to start small and see what happens.

##Family Holiday As always we Fosters are going to have a holiday in the summer and have some quality family time. I also want to take the boys to London for a weekend.

Blogging

This blog has taken a bit of a backseat in 2018 so I want to try and refocus it for 2019. My commitment is for 12 blog posts in 2019, that is one a month. It doesn’t sound like much but I know how easily life can get in the way.

Having said that my log has had record views this year. 7159 views. However I believe this number to be incorrect and has been inflated by my app development and some of the automated processes taking place.

Routine

Last goal is to try a get more of a routine going. With two small boys, one of which is starting school in September a routine is going to be essential and will allow all of us all a chance to get more things done.

Looking back at 2018

Thu, 27 Dec 2018 00:00:00 +0000

As 2018 starts to draw to a close let’s look at some of the highlights from the past year.

New Home - I started 2018 with the goal of buying my own home and I managed it. At the end of October I actually moved in but it was months of looking at houses, speaking to estate agents, saving, packing and unpacking. There is more I want to do to my home in 2019 so this is only really the beginning.
New Job - Another goal was to kick my career up a notch and this happened in June. It has been great working as part of a large team and I have learnt loads, there is much more I want to learn and contribute to so 2019 should be great on that front.
New Car - Not a goal that I achieved as unfortunately in October I was involved in a car accident, no one was seriously hurt but this was a horrible experience to go through. My car was written off which gave me an excuse to buy a new car.
See more of family was a goal, I am not sure if I really achieved this. However family have been around loads for us during our move and we do live closer so hopefully I can work on this some more.
Celebrate 5 years of marriage. It hasn’t been an easy year for me and the wife due to moving house and all the stress that involved plus with two children she was diagnosed with Postnatal Depression which as a family we are still dealing with. All that said we had a fab weekend away and we so need to do something similar again soon.
Family. 2018 has included many great times with my two boys. It’s been amazing seeing them grow, in September we had a thanksgiving service for them and in July we had our annual holiday.
Lightning Talk. This is the only thing I mentioned as a goal that didn’t happen, it’s on my 2019 list so hopefully I can kick myself into action but with everything else I achieved something had to slip.

Running SQL Server on a Linux Container using Docker for Windows

Mon, 05 Nov 2018 00:00:00 +0000

Recently I have been investigating what all the fuss is about Docker and it has been well worth my time as Docker is pretty awesome for automating stuff.

My development environment has typically required installing SQL Server. SQL is a bit of a beast with lots of options and takes time to setup how you want.

However since Microsoft have now created a version of SQL Server that runs on Linux you can run SQL Server in a Linux container with only a few commands.

I am going to assume you already have Docker for windows installed on your development machine. If not head over to Docker and find out how.

The Microsoft guide to setting up SQL Server in a Linux container can be found here .

First you need to download the image. In a powershell window run:

docker pull mcr.microsoft.com/mssql/server:2017-latest

This downloads the latest sql server image.

To run this image run the following:

docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=password"
-p 1433:1433 --name sql
-d mcr.microsoft.com/mssql/server:2017-latest

To run a SQL Server image you are required to accept the terms and conditions and set a default sa password. These are added as environment variables with the -e flag.

You also need to set the ports that your container will run on (1433 is the default SQL port) and give your container a name, in this case “sql”.

If you have already installed SQL Server you will not be able to run the container on the same port as your local install. To solve this you can select a different port.

docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=password" 
-p 1434:1433 --name sql
-d mcr.microsoft.com/mssql/server:2017-latest

-p 1434:1433 maps the 1433 port on the container to port 1434 of your local environment.

Once you have run this command you can connect SQL Server Management Studio (SSMS) to (local) or (local),1434 if you are using a different port using the credentials you provided and execute any SQL you like.

If your development environment requires windows authentication this of course is not for you, if it doesn’t you are good to go.

The development environment I have been using has various powershell scripts for setting things up. These assume windows auth. However I have adapted them to take custom credentials.

$credential = Get-Credential $server.ConnectionContext.LoginSecure=$false 
$server.ConnectionContext.set_Login($credential.UserName) 
$server.ConnectionContext.set_SecurePassword($credential.Password)

The Get-Credential command creates a dialog where you can enter SQL credentials, this is then stored in a variable and used in the rest of the script.

How do I restore a backup file to my container?

Run:

docker exec -it sql mkdir /var/opt/mssql/backup
docker cp database.bak sql:/var/opt/mssql/backup

This creates a backup folder and copies a backup file from your local environment to the container. You can then use management studio to restore the backup file (or you could write a sql script to do it). One thing to note when restoring databases, make sure the files are restored to Linux locations not windows locations.

The only issues I have encountered so far are the lack of support for SSIS packages and no windows auth. There are sql server windows images available which I haven’t tried yet which may work better with some of these options.

2018 The Story So far

Sat, 28 Jul 2018 00:00:00 +0000

Remember me? I used to write blog posts but somehow life got in the way. So much has happened this year, lets have a look at my goals for 2018 and see how far we have got with them.

Buy a house

My first goal for 2018 was to buy a house. Well so much progress has been made with this goal.

We looked at a few houses, and have made an offer, had it accepted and are waiting for the last few things to get arranged before we start arranging to move in.

The location we decided upon was Thorne . This is about an hour from York where most of our friends are and my wife has lived all her life, and also an hour from where my family live. I have tried my best to balance location and size of property and we think our new home is going to be a good balance but only time will tell.

I have a bit of a mammoth time ahead of me getting everything we own packed up and moved to our new home so wish me luck, but we will get there it is just going to be hard work.

New Job

In June 2018 I started a new job. This was slightly later than I had hoped but I believe it has been worth the wait.

My new job has lots of perks. Finish early on a Friday (1pm), On site canteen, Cool air-conditioned office, Flexible working hours, Part of a big development team.

The main benefit is that I am part of a big development team. I have a huge list of things to learn both in terms of technology and how large development teams function. The team is about to finish the first sprint that I have contributed to. My contributions have been very small. A bug fix here, a tweak of an API there. But the important thing is that I am making a contribution and learning along the way.

I have also started asking questions about deployment and how this could be automated. Something that interests me, especially with the sysadmin and devops background. I have been tasked with learning docker , so expect some blog posts on this topic in the near future.

See more of family

This one I am going to count as a miss. I have seen family a few times this year, but my efforts have been more on the first two items in this list. Hopefully this will change as things settle down.

Celebrate 5 years of Marriage

Me and the wife had a great weekend away to celebrate our wedding anniversary.

Celebrate my two children

Not happened yet but it is booked and we just need to arrange a few more details.

Lightning Talk

I have made no effort to do a talk and this is going to be a goal for 2019 I think. Most of the reason is concentrating on the other areas above, but also because I am not a natural speaker and I don’t like stepping out of my comfort zone. But I will do it, I just need to wait for other things to calm down a bit first.

Family Holiday

We had a great week away at the start of July. The weather was really great (and still is) and we had lots of time doing things as a family.

Casting and Converting between types

Mon, 07 May 2018 00:00:00 +0000

Recently I was asked how to convert a number to a string. Let’s look at a few ways of approaching this problem.

Most objects in c# have a method called ToString() which displays the string representation of that object. This is because of inheritance, all objects inherit from System.Object which defines ToString().

Int32 is a struct so it inherits from System.ValueType which also inherits from System.Object

so in code

int a = 9;
string b = a.ToString();

Now let’s look at the reverse. However the reverse runs the risk of throwing an error, let’s look at why.

string b = "9";
string c = "a";
string d = "two";

All are valid strings but only one can be converted to a number. Use the TryParse method to convert to a number.

int.TryParse("9", out int e);

TryParse will not throw an exception if the conversion fails, if it succeeds variable e will contain the result. Note an earlier version of c# required you to define the out parameter before using it with TryParse.

int.Parse exists to do the same thing however it will throw exceptions if a conversion is not possible. The same is true if you use Convert.ToInt32(“two”);

Casting

Casting is a way to explicitly telling the compiler that a type is actually another type and you are aware data loss will occur.

double x = 4.5;
int y = (int)x;

However it is not possible to cast a string to a number format as a string can contain any character not just number characters.

Let’s Encrypt is awesome

Mon, 30 Apr 2018 00:00:00 +0000

Let’s Encrypt is a free way to get a SSL certificate onto your website and until recently I had never tried it. It is very easy and I think it is awesome.

IIS is the web server software the Microsoft include with Windows 10 and Windows Server. I have it installed on my laptop and it displays the default IIS page.

It is not really a good idea to host websites on your laptop, use a dedicated web server, or host with a hosting company, however the techniques are the same and it gives me something to write about!

In order to point a domain name at what IIS on my machine was serving up I did the following:

Do a google search for “whats my IP”, this will return your public IP. Most residential ISPs use dynamic IPs so it may change over time, (which is another reason not to host a website on your laptop!)
Add an A record on a domain with the IP address you have just got
Your public IP most likely points at your router not your laptop so enable port forwarding of port 80 and port 443 to the internal IP of your laptop (something like 192.168.0.11 etc)

Now comes the fun Let’s Encrypt stuff!

First you need a Let’s Encrypt client, there are a lot of them out there mostly for linux flavours, however a bit of googling found a windows one. Go to https://github.com/PKISharp/win-acme/releases and download the zip file and unzip it.

Run the executable from the zip file and follow the onscreen prompts.

Press N to create a new certificate.

Then press 1 to bind to single website found in your IIS setup

And now magically Let’s Encrypt knows what you have setup in IIS.

Now all you need to do is enter an email address incase a renewal fails and agree to the let’s encrypt terms and you are all setup.

How awesome and easy is that for getting your websites working with a SSL certificate. If you have IIS configured on a server, give it a try and you can SSL all your things.

DNS for Developers

Mon, 09 Apr 2018 00:00:00 +0000

DNS is the backbone of the internet and as such I believe every developer should know something about the basics and not just leave it for the sysadmin to sort.

What is DNS?

DNS or Domain Name System is what translates Domain names to IP addresses and vice versa.

Wait what is an IP address and what are domain names?

You do realise this is a developer blog? An IP address is a unique address on the internet and a domain name is a user friendly label for one or more of these.

An example might be google.com which for me resolves to 216.58.204.14

How it works

When your browser makes a request to google.com it makes a request to your ISPs DNS Servers. This resolves google.com to 216.58.204.14

In more detail your ISPs DNS server will forward the DNS query to another DNS server and will cache the results for a set amount of time. This is the TTL or Time To Live. Next time the ISP DNS Server will be able to reply directly without needing to forward requests.

This forwarding and caching is what makes making a DNS change not instantaneous. The TTL needs to be reached so that no results are still being fetched from the cache of DNS servers across the globe.

DNS Records

Now we know roughly how DNS works let’s look at the most common type of records

A

A (Host) records are the most simple records which translate domain names to IPs

eg www.google.com to 216.58.204.14

CNAME

A CNAME (Canonical Name) record is different to an A record in that it maps a domain name to another domain name when no A record exists.

eg www.google.com to somethingelse.google.com

Typically Azure makes use of CNAMEs for many of its services especially adding a custom domain name

MX

MX stands for Mail Exchange and is used for configuring email

Name Server

Every domain has a number of Name Servers which tells you what servers control the DNS settings for that domain. If you change your Name Servers then the new Name servers will be where you can change your DNS settings.

If you want to use a service like DNSimple instead of 123reg or where ever you registered your domain then all you need to do is change your Name servers.

AAAA

Like A record but for ipv6

What next?

Want to put some different DNS records into practise? Buy a domain name and publish some content to it. Check out my previous post about programmatically adding records . Want an SSL certificate? Get a wildcard one and then you can apply it to any subdomain you add to your domain.

If you have a new website you want to publish consider which of the following is better:

https://www.example.com/newsite

https://newsite.example.com

I much prefer the second option, it looks cleaner, there is no potential conflict with the parent site, no subfolder issues between production and development.

Code Reviews

Mon, 02 Apr 2018 00:00:00 +0000

Reviewing code is a great habit to get into. Code reviews help share knowledge between your team members and help catch bugs before they get into production. But how do you get into the habit of reviewing and avoid the we don’t have time to do this mentality?

Visual Studio Team Services (VSTS) has some great options that can help make code reviews second nature.

Pull Requests

A lot of source control systems have the concept of pull requests. This is where you request others to review your code usually in a branch and if they approve it, merge it into a main branch.

To create a pull request in VSTS go to the Code section and select Pull Requests. Often VSTS will make a suggestion of what branch to make a pull request for, if you don’t see this just click the New Pull request button.

Select a branch you want to merge from and a branch that should be merged into (usually you merge into master from a feature branch). Give your pull request a title and description and select who should review your code, this can either be an individual or a group of people. You can also review all the changes that will be reviewed so you can make any last minute changes before it is reviewed.

Now if you are anything like me you want your code merged in as soon as you have created your pull request and there is nothing stopping you reviewing your own code and clicking approve and merge on your own pull request. However branch policies is a way around this problem.

Branch Policies

Branch policies allow you to specify how your code gets merged in.

Go to the list of branches in VSTS and select branch policy and you will see a whole host of options to customise the merge process. If you do this on the master branch you will not be able to commit any changes to master without it going through a pull request.

The first option enables you to select how many reviewers are needed on your code. If no one else works on your project best not setting this, but for everyone else setting at least one person to review your code is a great practice.

Next you can ensure that your pull request is linked to a work item, this helps keep ensure you are actually fixing issues and not just making change for the sake of it.

Check for comment resolution is a good setting to enable. This ensures that if your reviewer has commented about you needing to change this line here, it ensures that you do.

Enforce merge strategy allows you to choose between fast forward merge or squash merge.

Build validation enables the code to be built using a build definition you have configured. This is a great way to check code builds or tests pass before it gets merged in.

The last two options allow you to specify code reviewers and third party external services.

Android Development Options

Mon, 26 Mar 2018 00:00:00 +0000

A friend asked me how to get started in Android Development and I thought I might have a go at answering that question here.

I am by no means an expert in Android development, I do have an app in the play store so I know something.

Manifest File

This is probably the easiest option and also doesn’t actually create an android app so I am not sure if it should be included in this list of not.

If you have a website and you want to create an app for that you could just create a manifest file and add this to your website.

Once your website has a manifest file, if you visit your website using a mobile phone or tablet you will get the option to add a shortcut to the home screen. You then have an app like experience in that you can click an icon to launch your website.

A manifest file is a simple text file which specifies a few settings like the icon size, filename, what page loads when clicked and name of your “app”

More information

Visual Studio

This is the option I know most about as is what I have used.

If you are familiar with Visual Studio you can use the Xamarin Forms software to create your app in C#. Xamarin Forms allows you to easily create cross platform apps that run on Android, IOS and windows phone. So far I have only experimented with Android but it should be relatively easy to extend my code to run on other platforms.

Xamarin Forms allows you to write one a single codebase that can be compiled to run on the different platforms. Xamarin requires the use of XAML a XML like markup language for designing UI elements.

More Information on Visual Studio , Xamarin Forms

Android Studio

I don’t know much about this option so do correct me if I don’t get the details correct.

Android Studio can be downloaded from Google this allows you to create java code to run directly on an android device. From what I know this is fairly similar experience to Visual Studio but instead of writing your code in C# you use Android Studio and write it directly in java.

More information

Cordova

Cordova allows you to use HTML, CSS and javascript to create cross platform apps. I have no idea why I haven’t heard of this technology until today as it sounds very flexible especially if you know a little bit of javascript.

More information

To summarize there are lots of different options available to create an android app. What you choose depends on what you want to build, what language and experience you have and if your app needs to be cross platform.

Tips for Developing Yourself

Mon, 19 Mar 2018 00:00:00 +0000

For a while I have been mentoring a friend and I thought I might share some top tips I have implemented in my career so far.

Hard Work

At school I was told: “Only in the dictionary does Success come before Work“. Yes it is a bit of a corny saying however it has stuck in my head and it’s true. If you want to get anywhere you need to work at it. For many years I just drifted along, but as soon as I knuckled down and worked hard my career started going somewhere.

Write a Blog

Writing is an important skill that you need to work at to improve. If you commit to writing one blog post a week this will help to improve this skill. Even after 150+ blog posts I still feel I need to improve. Not only that it will also help others learn something from you and shows a willingness to contribute to the community. If a Hiring Manager reads your blog he can see some of the stuff you know or care about and may help give you the edge over more experienced candidates.

Soft Skills

This is one I need to work on but you should concentrate on the so called “Soft Skills“. Soft Skills are the skills you have for dealing with others. Are you good at talking to people? Or good at getting requirements out of business owners? If so you have some soft skills, don’t underestimate the value that these bring to a team and keep developing them.

Build a Side Project

Start building a side project, it doesn’t matter what it is but start building something. I started building a Xamarin app but I have learnt loads of other connected things while doing this, ViewModels, APIs, Build and Deployment processes, Azure Functions. It also gives you ammunition for things to blog about. If you are short of ideas try solving a problem or rebuild something you use.

Use your spare time

During the day there are moments you can reclaim for learning stuff. There are lots of podcast which discuss useful development topics, listen to these while driving to work. Subscribe to pluralsight and listen to this while washing up. Don’t get too concerned with learning everything, however think how much more you are learning than not listening at all.

I’m not ready!

We are never ready to take that next step, it is so easy to make excuses like I need to learn x, or know y. The only way to know if you are ready is to try and to keep trying. Iterate as you go so you keep improving yourself. If you wait before you start trying you will end up waiting forever. If you start trying now you will make some small progress and have a better idea of what you need to do to achieve your goals.

Heroes

Fri, 16 Mar 2018 00:00:00 +0000

Who was your hero growing up? Mine was Stephen Hawking.

What is a hero? A lot is said in the media about “Heroes”. Often the sportsperson of the hour is described as a hero. I don’t like this definition and I think there is far more to being a hero than that. One definition I saw describes a hero as A person who is admired for their courage, outstanding achievements, or noble qualities.

Why Hawking? As a kid I thought I was pretty smart (Not really just above average), I liked Star Trek and science so Hawking was a pretty obvious choice. I remember reading all about Hawking and his life fascinated me. A brilliant man was struck down by motor neurone disease in his early twenties, however this did not stop him and he completed his doctorate and became the famous physicist we all know and love.

My physics career was fairly short lived ending with an undergraduate course at University of York, however I really enjoyed learning the concepts that he described about space and time and was definitely a driving factor for me to learn as much as I did about Physics.

How heroic is it to be given 2 years to live and then not only survive but to expand our understanding of the universe without being able to speak or move without the aid of technology?

I can not imagine what it must be like to not be able to communicate without the use of a computer, but this has been Stephen Hawkings world for over 30 years. His computer system has undergone various upgrades over the years however his synthetic computer voice has continued as he identified with it and prefered it.

Stephen Hawking had an immense intellect, I would probably describe him as one of the cleverest people on the planet.

Who are your heroes? What kind of person is your hero, is it a scientist or engineer? The world contains some very clever people who will inspire and excite young people to strive towards something. I am hopeful that my sons can be inspired even if it’s only a small way by someone like Stephen Hawking.

New version of Pwned Pass

Mon, 05 Mar 2018 00:00:00 +0000

A new version of Pwned Pass is available from Google Play .

A couple of weeks ago Troy Hunt released V2 of Pwned Pass onto his haveibeenpwned website. There are now over half a billion passwords that have appeared in data breaches for you to search.

This time Troy has included information about how many times a password has appeared in his data. So “password” returns a value of 3,303,003 so is a really, really bad choice of password to use. However the password “windows 10” has only appeared twice so is much better, however I would still recommend avoiding using it.

One of the great things about the internet is that on the day of release I was tweeting Troy so I could get my android app updated to take advantage of these new features. I even helped him identify an issue with the documentation on his site. He has also kindly add a link to my app onto his website.

That conversation and link has lead to a massive increase in app downloads. Before I was going steady on about 14 installs, now I have 86 active installs. I am not sure if this will increase anymore but if you are reading this because you downloaded my app, then thank you.

My app has four main features:

Password Check – this allows you to search a password and see if it has appeared in a data breach and also the number of times that password has appeared.

Email Check – this allows you to search your email address and see which data breach it has been involved in.

List of Data Breaches – this lists the data breaches from haveibeenpwned.com you can also sort by Date Added, Number of accounts and name.

Calendar of Breaches – this shows a github like chart of what days breaches are added

If you like my app do let me know. I have received a few ratings on google play it would be great to get a few more. If you want me to add a feature or have ideas of how I could improve it let me know as well.

To download take a look at https://play.google.com/store/apps/details?id=pwnedpasswords.pwnedpasswords

Refactoring if statements

Mon, 26 Feb 2018 00:00:00 +0000

The code base I am working on contains a huge if block. By huge I mean 77 if statements one after the other, each if checks to see what page id you are on and loads different content. This is not easy to maintain and I want to refactor it.

One option would be to replace the if statements with a switch block. However this is just as unmanageable as the huge if block. Lets look at a better option.

Polymorphism is where you create a base class and then create sub classes from it. In my case I created an interface IPage with a single method CreateContent.

public interface IPage
{ 
  string CreatePageContent(); 
}

and then create 77 classes for each page which implemented this single method.

Now comes the fun bit how do I call the correct page class from my original code?

I created a dictionary than maps page ids to the class names.

public static Dictionary<PageIds, Type> PageIdToClass = new Dictionary<PageIds, Type>() 
{ 
  { 
    PageIds.HomePage, 
    typeof(HomePage) 
  }, 
  { 
    PageIds.ContactPage, 
    typeof(ContactPage) 
  }, 
  //etc 
}

This is the one step I am not 100% happy with as I think it may be possible to remove or simplify this step.

Now I have a way to map ids to classes I can write a class to do this.

public class MyPage 
{ 
  IPage _repo; 
  
  public MyPage(int pageId) 
  { 
    PageIds p = (PageIds)pageId; 
    Type t = PageIdToClass[p]; 
    ConstructorInfo constructor = t.GetConstructor(new Type[] { }); 
    _repo = (IPage)constructor.Invoke(null); 
  } 
  
  public string Create() 
  { 
    return _repo.CreatePageContent(); 
  } 
}

So in my constructor I take the pageId and pass it to my dictionary to get which subclass to load. I then get its Constructor and invoke it.

Now I can remove the huge if block and replace it with a single line of code.

var page = new MyPage(pageId);

On the face of it this change might look like a lot of work for not much gain as we started off with one file and now we have the original file, an interface, 77 subclasses and the MyPage class. However the original file is a lot more manageable and each sub class can be altered independently of each other.

This is a big step towards making this code more maintainable, there is always more that can be done but that can wait for another day.

Chrome distrusts SSL Certificates

Mon, 19 Feb 2018 00:00:00 +0000

One of the websites I have been working on has been displaying an error in the console. The error reads as follows.

The SSL certificate used to load resources from https://example.com will be distrusted in M70. Once distrusted, users will be prevented from loading these resources. See https://g.co/chrome/symantecpkicerts for more information.

But what does this mean? Well let’s start by looking at the link provided.

In January 2017 it was revealed that Certificate Authorities run by Symantec which include Thawte, VeriSign, Equifax, GeoTrust, and RapidSSL had been issuing certificates that did not comply with baseline standards.

Starting with Chrome 66, Google has decided to remove trust for these certificates. Chrome 66 is due for release around 17th April. My error mentions M70 so what does that refer to?

Chrome 70 which is due to be released in October 2018 will removed the trust for another batch of Symantec certificates.

If you are getting one of these errors because you are using a certificate that is going to be distrusted what will your site look like in Chrome 66 or Chrome 70?

Well Chrome 66 is now in the dev channel so we can give it a try.

Not very nice for your users is it? Now is the time to order a new SSL certificate to avoid this happening to your site.

I first saw this error a few months ago and have been reading up about it and waiting for Chrome 66 to reach the dev channel so I could test what it did to my site. However now that I have Chrome 66 installed I spotted the intranet for the company I work for is also affected. I do not directly work on the intranet so I notified the security team that they may want to look into this.

Unfortunately the response I received has been that Google needs to fix this before Chrome 66 is released. I am not criticising my employer or the security team, however this isn’t something Google can just “ fix “.

The certificates issued were issued by a CA that had issues so in order to maintain the trustworthiness of all certificates Google had little choice but to distrust them. Google and security experts need to be making more of a fuss about this and I am joining in on making a fuss by writing this blog. Scott Helme estimates that there are about 7000 websites which may be affected by the M66 and M70 distrusts.

Content Security Policies

Mon, 12 Feb 2018 00:00:00 +0000

A content Security Policy or CSP is a HTTP response header that defines what sources of content can be loaded on a web page. It is a way to combat Cross Site Scripting (XSS) attacks.

What is a XSS attack then?

When you load a webpage it also loads various other resources like images, some css style sheets, various javascript files that you want to run and probably many other things.

How do you know that you can trust all of these things? If you created them and they live under you control then the answer is probably yes. However these days you will probably want to use resources from across the internet, like youtube videos, google analytics, disqus comments, jquery libraries from a cdn etc and you can’t be sure exactly what they are doing.

Imagine you had a page which you could add any text into a form which would then be displayed. A malicious user could add evil javascript or get the browser to load evil code from anywhere on the internet.

CSP to the rescue!

A CSP allows the browser to only load from sources that you specify. You could specify that resources from your own site will load but the evil script will not.

Let’s look at some examples

Content-Security-Policy: script-src 'self'

This allows <script> tags to only load from the current webhost. script-src is not the only keyword you can use, let’s look at some of the others.

script-src – control what <script> tags will load
style-src – control what css will load
img-src – control what images will load
frame-src – control what frames will load
font-src – control what fonts will load
object-src – control what object tags will load
connect-src – control what resources a script can connect to
media-src – controls what media (audio/video) will load
default-src – if no specific rule exists then the default directive will run

Content-Security-Policy: default-src https

This allows any content to be loaded from any site as long as it comes from a secure (https) site

Content-Security-Policy: default-src https://example.com

This allows any content to be loaded from https://example.com only.

How do I use this on my site?

I have added CSPs into my web.config which works great for my .Net Framework code.

 <system.webServer> <httpProtocol> <customHeaders> <add name="Content-Security-Policy" value="default-src https://example.com" /> </customHeaders> </httpProtocol> </system.webServer>

For .net core it is a bit more complex as you don’t tend to use web.config files, however check out Anthony Chu’s post , which has a solution to that problem.

Report Only

One last thing about CSPs to mention is the Report Only flag.

Content-Security-Policy-Report-Only

This does the same as the above but doesn’t enforce anything, so you can fix any problems before you break anything.

To view your issues just look in the developer tools in your favourite browser. Or you can configure all your reports to be collated in one place with a report-uri directive.

Content-Security-Policy: default-src https://example.com; report-uri https://example.report-uri.com/r/d/csp/reportOnly;

Scott Helme and Troy Hunt have a site called report-uri which offer a service for collating and viewing all your CSP violations so check it out if you want to know more about CSPs.

Pluralsight

Mon, 05 Feb 2018 00:00:00 +0000

Earlier this year I signed up for pluralsight. If you want to sign up to pluralsight as well use this link http://referral.pluralsight.com/mQd8BJ4 to get money off.

Pluralsight is a website that sells training videos on a wide variety of technical topics. You sign up for a monthly or annual subscription and you can watch over 6000 courses whenever and where ever you like.

I am not a big fan of videos as if I am sat in front of a screen I would rather be building something, however I do spend a lot of time listening to podcasts. So I have decided to convert this time to listening to pluralsight videos.

If we assume that by not seeing the visual portion of the videos you lose out on 75% of the learning (This is just a guesstimate I am not sure what the exact figure is or how you would calculate it). However this is still 25% more learning than if I hadn’t signed up at all.

Since I signed up I have listened to over 27 hours of technical videos. These have covered topics like C#, Security, Xamarin, MVVM, Interfaces, Getting Involved and Clean architecture. The number one thing I have learnt is that there is a lot that I still don’t understand.

That said I would like to think that since starting to listen to pluralsight concepts have gone from being unknown unknowns (I don’t know about them and I don’t understand them) to known unknowns (I know about them but don’t yet fully understand them)

To keep track of your learning pluralsight has some tests which you can complete. I am particularly proud of my score on the Azure test which ranks me as an expert.

Why not check out what you can learn from pluralsight?

Flexible Architecture with Interfaces

Mon, 29 Jan 2018 00:00:00 +0000

I have blogged a few times about interfaces , and how useful they are for producing good quality maintainable code. Let’s look at a problem and the solution I came up with which I am quite proud of.

As previously mentioned I am in the process of moving images from AWS to Azure blob storage. Now that the actual files themselves have been moved I need to change the code that references them.

Now I could find all the code that uses the AWS API and replace it with the Azure API but I am not very good at predicting the future, we may stay on Azure for a while, we may move to AWS or Google Cloud, or we may want to go back to files sitting on a server.

Lets try and code a solution that is as flexible as possible. As you have probably guessed I am going to create an interface.

At first I thought about creating an interface called ICloudStorage , however this isn’t flexible enough as what happens if we go back to sticking files on a server so instead I created IStorage.

I created three classes that implemented IStorage, AWSStorage , AzureStorage and mostly for testing at the moment FileStorage. I then created a class Storage that would call these three classes. Initially I created it like this

public class Storage
{
  private IStorage _repo;
  public Storage(IStorage repo)
  {
    _repo = repo;
  }
}

However this would require I call it like Storage(new AzureStorage()) and I would need to know everywhere in my code which implementation I want to use. This isn’t too bad as when we change from AWS to Azure we would need to do a find and replace throughout the code and replace all AWSStorage and make them AzureStorage.

However we can do better than that.

public class Storage
{
  private IStorage _repo;
  public Storage()
  {
    Type obj = Type.GetType(ConfigurationManager.AppSettings["DefaultStorageRepository"]);
    ConstructorInfo constructor = obj.GetConstructor(new Type[] { });
    _repo = (IStorage)constructor.Invoke(null);
  }
}

This code will read from the web.config which implementation to use and that will decide which class to call. This means that to change from AWS to Azure we do not need to redeploy any code, all we need to do is change the web.config.

Let’s look at the three lines and see if we can understand what is happening.

Type.GetType() looks straight forward and gets the type from the web.config

obj.GetConstructor() This gets the constructor for the type we have just found.

constructor.Invoke This then invokes the constructor and it then gets cast to the interface so can be used by the _repo variable.

This is all fairly simple and makes sense, however it has produced some very flexible code and allows the code to be extended without recompiling.

Let’s look at a hypothetical example. We want to add support for Google Cloud Storage. All we need to do is create a class library which implements the IStorage interface, place the compiled binary in the website and update the web.config to reference it. I haven’t tried this hypothetical example so it might be more complex than I think but in theory it should work.

I am pretty excited at how flexible this code can be, hopefully I will use code like this more often now I understand it.

Moving files into blob storage

Mon, 22 Jan 2018 00:00:00 +0000

We are in the process of moving our companies websites onto the Azure platform. One of the challenges was to move image files out of the website project into blob storage. This week I have moved 150,000 of them.

One thing I keep banging on about is that your source code should not contain data. If it does every time you do a deployment you need to consider where these images are located and ensure you don’t overwrite or loose any. It also goes without saying that deployments of a few Mb are a lot quicker than deployments of 100s of Mb.

Azure blob storage also gives you advantages like distributing storage across multiple datacenters which would be impossible with traditional files on a server.

So now that we have established that this is a good idea lets look at how we could move large amounts of data. In my case all the filenames are stored in a SQL database so the plan of action was to simply loop through the files in the database, download from current storage (either locally or other cloud storage), upload to Azure and tidy up afterwards. Due to the number of images I am going to update the database and mark when a file has been processed so I can do the move over several days.

This is my code

var source = "https://example.com/images/";
var tmp = Server.MapPath("~/tmp/");
if (!Directory.Exists(tmp))
{
 Directory.CreateDirectory(tmp);
}
var fixturePhotos = db.Images.Where(x => x.Moved == null || x.Moved == 0).Take(id);

foreach (var photo in fixturePhotos)
{
 try
 {
  string path = getFilePath(photo.FileName);

  if (!Directory.Exists(Server.MapPath("~/tmp/" + path)))
  {
    Directory.CreateDirectory(Server.MapPath("~/tmp/" + path));
  }
  WebClient WebClient = new WebClient();
  WebClient.DownloadFile(source + photo.FileName, tmp + photo.FileName);
  FileUploader f = new FileUploader(tmp + photo.FileName, photo.FileName);
  System.IO.File.Delete(tmp + photo.FileName);
  photo.Moved = 1;
 }
 catch
 {
  photo.Moved = 0;
 }
}
db.SaveChanges();
if (Directory.Exists(tmp))
{
 Directory.Delete(tmp, true);
}

First of all I create a tmp folder in the root of my website if it doesn’t exist to store my images temporarily.

I then use an entity framework model to query the database that haven’t been moved, and I use the take() method to limit how many results I process. (I have been passing in 1000 at a time)

I then use a foreach loop over all these files to perform the following actions.

Create additional subfolders if the filename variable stored in the database isn’t actually a filename but a filepath, note you will have to split filename and filepath which I haven’t included code for here.
Download file from the original url and save into the temporary folder
Upload to Azure
Delete temporary file
Update database giving a success or fail

Once the foreach is finished I commit the database changes and delete the temporary folder. I am sure there must be other ways to do this transfer but this was quick and easy to setup and now I have a copy of all the files in Azure storage so I can test out other issues with my website.

One last tip about how to schedule this code. I called the above code from a MVC controller and then wrote a Azure Function to call this code on a schedule.

Website UI Testing

Mon, 15 Jan 2018 00:00:00 +0000

Last week I looked at testing the UI of mobile apps , this week lets look at how we could do a similar thing for websites.

Testing the user interface is not an excuse for a lack of unit tests . Testing the user interface takes longer so for keep creating your small unit tests that can be run after ever build. That said lets look at how you create a UI test.

Create a Unit Test project as normal. Now install the following nuget packages

Selenium.WebDriver.ChromeDriverSelenium.WebDriverSelenium.WebDriver.PhantomJS.Xplatform

I am going to be using Selenium to achieve my website testing and I am going to concentrate on the Chrome browser. However packages exist for other browsers so have a look at the following and I expect there are others as well.

Selenium.WebDriver.IEDriverSelenium.Firefox.WebDriver

Selenium started life as a plugin for Firefox to help create automated tests, however the latest version of Firefox is not compatible with the plugin as I write this. I have not had to install any plugins or extensions to my browsers to achieve my testing.

Enough talk lets write a test. First we create two instance variables to store the baseURL and the driver for the browser you are using.

private string baseURL = "https://www.example.com/";
private RemoteWebDriver driver;

Next we need to set things up for the test to run. This creates an instance of the chrome driver, maximizes the window and sets it to wait 30 seconds before timing out.

[TestInitialize()]
public void MyTestInitialize()
{    
  driver = new ChromeDriver();    
  driver.Manage().Window.Maximize();    
  driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(30));
}

Now comes the actual test. We navigate to a URL and then compare the title of the page loaded with a know value with a Assert statement like you would find in a unit test.

[TestMethod]
public void CheckBrowserTitle()
{    
  driver.Navigate().GoToUrl(this.baseURL);    
  Assert.AreEqual("Home Page", driver.Title);
}

Finally we need to tidy up after ourselves.

[TestCleanup()]
public void MyTestCleanup()
{    
  driver.Quit();
}

If you are used to writing tests you will know that the are usually constructed in three sections Arrange, Act and Assert. The Arrange is done in the initialize method, which makes the actual test much simpler, the first line does the Act and the last line does the Assert.

Now we have written a simple test lets look at something more complex.

driver.FindElementByLinkTest("click")

This finds any Link on the page which is click. Be careful as the string need to be exactly what appears on screen it may well be easier to specify by id or class or something that doesn’t change as often. By adding .click() on the end of this command Selenium will click on the link and you can navigate to a new page.

What about submitting form data? Well you can find the element you want to fill in and add .SendKeys(“example text”) or .Submit() and this will fill in and submit form data.

What about a screenshot?

Screenshot ss = driver.GetScreenshot();
ss.SaveAsFile("test.jpg",System.Drawing.Imaging.ImageFormat.Jpeg);

I have only just started playing around with web UI tests but you can see there is a fair bit you can do.

Mobile App UI Testing

Mon, 08 Jan 2018 00:00:00 +0000

Since I started creating an android app I have been writing simple UI tests.

I have been taking advantage of the Visual Studio App Center which allows you to test against hundreds of different devices in the Test Cloud.

In order to write a UI test create a UI Test App, this makes use of the nuget package Xamarin UI Test. By default you will now have a test called AppLaunches which will take a screenshot of you app after it starts.

You can now run this test against any device from Visual Studio assuming you have it physically plugged into your machine. However, how do you run against the Test Cloud?

To run against the Test Cloud you need to first install node.js. Now you can install the appcenter cli with the following command.

**npm** install -g appcenter-cli

To run the tests in the Test Cloud run the following command

**appcenter** test run uitest --app "<username>/<appname>" --devices "<username>/<deviceset>" --app-path _pathToFile.apk_ --test-series "master" --locale "en_US" --build-dir _pathToUITestBuildDir_

where is your appcenter username, is the name of your app in appcenter and is the group of apps you have created in app center to test against.

Look at device sets in the test section of the appcenter and click the new device set button. You can then search for any device you like and add it to a set, as I write this there are 244 devices you can test against.

It is currently not possible within app center to run tests against a new build, however if you build your app in VSTS as well you can create build or release step that runs it. As the Visual Studio app center is still under development I wouldn’t be surprised if it is added at some point.

In VSTS look for Mobile Center Test in the definitions and you can specify the same variables as specified in the command line above.

Now how do you actually write a useful UI test? I mentioned above you get a default test which contains the following code, this takes a screenshot of your app. However you don’t have to include this when using the Test Cloud as screenshots are included for free.

app.Screenshot("First screen.");

Lets look at what else is included in app

app.Repl();

This starts an interactive REPL (Read-Eval-Print-Loop) which lets you explore what is on screen in your app and pauses execution. I don’t include this in my tests, however I do make use of it to explore what is on screen and what tests I might make use of.

app.Tap(c => c.Marked("Button"));

This taps an element on screen called Button. There is also a method called TapCoordinates which would allow you to click anywhere you like.

app.WaitForElement(c => c.Marked("View"));

After clicking a button you are probably going to want to wait for the app to load extra data or a new screen. The WaitForElement waits for an element to appear on screen. There are also methods that wait a period of time or wait until an element no longer exists.

These are the main methods I have used so far, however there is an extensive list including methods for scrolling, swiping, pinching and adjusting the volume buttons. So you should be able to test all manner of app functionality and if you make use of the Test Clouds will know which devices are causing problems.

Lets see what 2018 can do!

Mon, 01 Jan 2018 00:00:00 +0000

It is 2018 so it must be time to think about what my plans and goals are for the new year.

Buy a house

This a huge goal for me. For many years I was content with renting and thought owning my own home wasn’t for me, but then I went and started a family and my career started going in the right direction and my thoughts about this have changed.

It is going to take a lot of work to make this goal happen in 2018. First I need to save enough money for the deposit, which is far from easy with a wife and two children. Second I need to get our current place sorted ready to physically move which again won’t be easy. Third I need to make a decision which will affect where I live for the next 25 years. At the moment I am torn between the size of property and its physical location, I need to balance both of these and meet the requirements from the rest of the family.

I do think this is an achievable goal so wish me luck.

Progress in my career

I need to be careful how I phrase this in case past, current or future employers are reading. I have been working hard at my current role for over a year and I am starting to wonder what is next for me. I am not entirely sure what form this is going to take yet, I have a few ideas on the go and hopefully I can reveal more soon.

Another one that is very achievable, as long as I have it in the back of my mind as the year goes on progress should be made.

See more of family

I recently missed a family celebration due to the expansion of my own family and it reminded me that there are aunts, uncles and cousins that I haven’t seen in years and it would be nice to reconnect. Also as my own boys get older and are learning to interact it would be great if they could get to know my own family better.

I don’t have a clear idea how to achieve this one as it takes two but I am going to make more of an effort.

Celebrate 5 years of marriage

Wow how has it been five years since we got married? At some point in 2018 I will take time out from the usual and spend it with the wife away from the boys. I won’t give away more details as I want that to be a surprise.

Celebrate the birth of my two sons

I don’t want to get my boys christened as I would like them to make up their own minds when they are old enough, however I do want to celebrate all that they are with a thanksgiving service. Not sure when we are going to sort this, however it is my hope that it will be soon.

Family holiday

Not much of a goal but in the summer I will take time out to spend a week away with the family. In previous years we have done scotland, wales and northumberland. This year I am thinking of somewhere not too far away as Edward is still small, but maybe near where I grew up.

Lightning Talk

A lightning talk is a very short talk or presentation given at a conference or user group. Friends of mine are having lots of success talking about what they know about and I think it might be time for me to have a go. I am not a natural public speaker so this is a step out of my comfort zone, however I am going to start small and see what happens. At the moment I don’t know what my subject or topic is going to be, my suspicion is that it will be devops related as that interest me more but watch this space.

Looking back at 2017

Thu, 28 Dec 2017 20:00:45 +0000

As 2017 starts to draw to a close let’s look at some of the highlights from the past year.

Edward

In November I became a father again and celebrated the arrival of Edward into the world. Before the birth I was worried how well my first son James would adapt to the new arrival but I am happy to say I didn’t need to, he mostly ignores the new arrival.

Star Trek

In September the latest Star Trek TV series Discovery arrived on Netflix. I have been very positive so far about the show, I am still positive however there are things that annoy me about the new show, however I am keen to see the next episode each week so they are definitely doing something right.

Office move

In May I helped my employer move to brand new offices . This was a lot of hard work, but everything went as planned and our new home is great.

Side projects

I blogged in February about having a side project but it wasn’t until later in the year that this solidified into what I am working on now. It started as a way to understand interfaces this then led to learning Xamarin and creating an android app to help promote this blog. Troy Hunt expanded his HIBP API to include passwords that have been in data breaches, I then started working on pwned passwords which is a simple android frontend for this.

Learning R

In Feb I also experimented with a new programming language R . It started with a problem about converting exchange rates and ended with a SQL stored procedure that executed some R code, I still want to learn lots more but it was a good example of the sorts of thing you can build with R.

DNS programmatically

One of my most successful blogs was about how you can create dns records within your application eg sites that have yourname.domain in the url. It was a great chance to look at how dns works and learn more about what azure can do to manage this. I want to expand on this and build something.

So that was 2017, for some ideas about what 2018 might have in store for me look out for my next blog post.

Getting started with Azure Table Storage

Sun, 17 Dec 2017 20:00:45 +0000

Azure Table storage is cheap way to store data, however it has some drawbacks that you should be aware of.

Azure Table storage is a simple way to store NoSQL data with key/attribute pairs. I am very familiar with storing data in SQL databases and would still choose SQL over Table storage, however Table storage is significantly cheaper so could be worth investigating depending on your project.

Troy Hunt makes use of Table storage for his Have I been pwned? website so there are projects out there that make use of it to great affect.

To work with Table storage you need to use a nuget package WindowsAzure.Storage

Install-Package WindowsAzure.Storage

To load data from a Table in Azure Table storage I use the following code

CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("tablename");
TableOperation retrieveOperation = TableOperation.Retrieve<Entity>(PartitionKey, Rowkey);
table.CreateIfNotExists();
TableResult retrievedResult = table.Execute(retrieveOperation);
 
Example eg = new Example();
if (retrievedResult.Result != null)
{
 eg.ID = ((Entity)retrievedResult.Result).Id;
 eg.Date = ((Entity)retrievedResult.Result).Date;
}
return eg;

You need to create a class (called Entity in the example above) derived from TableEntity which defines the Partitionkey and Rowkey, plus and other columns you want to store in table storage. The row key and partition key uniquely identify the data in the table, think of this as the primary key of the table if you are used to SQL. This class must also contain a parameterless constructor.

This is the only way to retrieve data, using the partitionkey and rowkey. If you want to retrieve a specific piece of data you would need to retrieve all rows and then search them for what you need. For me this is not a big problem as I only have 150 rows but if you have millions of rows you may need to think carefully how to use this.

To save data I use a very similar piece of code

CloudStorageAccount storageAccount = CloudStorageAccount.Parse(path);
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("tablename");
Entity post = new Entity(ID, DateTime.Now); 
TableOperation insertOperation = TableOperation.InsertOrReplace(post);
table.CreateIfNotExists();
table.Execute(insertOperation);

I create a Entity object and then pass this as a parameter into the InsertOrReplace method.

To delete data it is also very similar, you create an entity object and pass this as a parameter to the Delete method.

When debugging my table storage code I found the Azure Storage Explorer very useful for seeing what data actually existed in the table and what might be throwing an error, usually something wrong with my Entity.

I mentioned earlier that Table Storage was cheaper than SQL Azure. Well for my simple playing about with things I have found my monthly charge of £10+ has been reduced to £1+ If I were to build anything that is more than just me learning about how it works I would probably continue to use SQL but for the cost of learning new tech it is well worth giving table storage a try.

How to get Zero Code Changes outside of Source Control

Mon, 13 Nov 2017 20:00:45 +0000

Source control is bread and butter for web developers, however not so much for SQL developers and other business people. One of my goals for the coming year is to get the whole of my team using source control processes.

One of the arguments against using source control is there will always be a few exceptions where it won’t be used. Lets look at a few scenarios to help make the case for source control.

Scenario One

Web Dev 1 makes a change in production to fix an issue and doesn’t use source control. Web Dev 2 is working on a new feature and makes use of source control. He is given the sign-off to deploy his new feature and in the process undoes Web Dev 1’s fix.

Management blames Web Dev 2 as they “broke” stuff during the deployment. Web Dev 2 doesn’t understand what happened and Web Dev 1 is oblivious to the entire scenario despite in reality being part of the problem.

Scenario Two

A SQL Dev works all night to fix a problem. He deploy to production and doesn’t notice a missed where clause due to lack of sleep.

—

Let’s look at these two scenarios done with the whole team buying into source control.

Scenario One

Web Dev 1 makes a change in production to fix an issue and also commits to source control. This takes him an extra 5 minutes (if that!) Web Dev 2 is working on a new feature and makes use of source control. He is given the sign-off to deploy his new feature and in the process merges Web Dev 1’s fix before he deploys to production.

Management is happy with Web Dev 1, nothing is broken and new functionality has made the website better not worse.

Scenario Two

A SQL Dev works all night to fix a problem. He deploy to production and doesn’t notice a missed where clause due to lack of sleep, he also commits to source control this takes him an extra 5 minutes (if that!).

In the morning a colleague looks through the commits made during the night and queries the missing where clause. A fix is made in source control and deployed, management and the client are unaware that a problem was created and fixed.

—

These are simple scenarios but I cannot think of any situation where the outcome would be better to not use source control. These examples assume no continuous deployment, adding this to the system before every team has bought into source control would cause bugs and issues all over the place.

However with a team that is 100% behind source control continuous deployment can achieve an amazing productivity boost. After every commit code could be automatically tested, reviewed by other members of the team and deployed to test environments for further analysis. Only tested code that has been reviewed can get anywhere near production environments.

Having said all this there are ways especially on the SQL side to mitigate loosing changes while you work on training individuals and convincing management. Before any code is deployed to production do a schema compare. If you only see changes you have made you can proceed, if you don’t shout at your team.

Advantages

Code Reviews
Team awareness of what is being changed
History of changes
Smoother Deployments
Automation
Better Teamwork
Diff changes between different versions
Many more

Disadvantages

Time to learn tools
Remember to commit changes
Technologies that use binary files harder to version control

SQL with Visual Studio Code

Mon, 06 Nov 2017 20:00:45 +0000

Writing SQL queries is typically done with SQL Management Studio (SSMS). However this tool is a bit of a beast so let’s look at how you could use Visual Studio Code instead.

Visual Studio Code is a free text editor but it is so much more than just a text editor. VS Code can be downloaded from https://code.visualstudio.com/Download

To work with SQL Server download the mssql extension. Press CTRL+SHIFT+P and then Select Install Extension and type mssql.

Intellisense in Visual Studio Code is brilliant, better than SSMS. Lets look at how to get it all set up.

Create a new file and set the language type to SQL (Press CTRL+K,M )

Open the command palette, CTRL+SHIFT+P and type SQL to show the mssql commands. Select the Connect command.

Then select Create Connection Profile , this creates a profile to connect with your SQL Server. Follow the prompts to get it all setup.

Look in the bottom right corner of the status bar and you should see you are connected.

Now if you type sql you will see a long list of SQL code snippets that you could use.

Choose a snippet to create, and edit it as required. When you are happy press **CTRL+SHIFT+E ** to execute.

This is basically all there is to it. However this is an incredibly powerful way of working, the intellisense instantly tells you what database objects you can use in your query and there is a wealth of different snippets you can use.

When returning data you get a similar view to SSMS but you can save as Excel, CSV or JSON.

SSMS is a very graphical way of doing things, you can double click a table and see its columns or indexes. VS Code relies on TSQL commands but you have access to exactly the same information.

For more information about VS Code and the mssql extension check out https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-develop-use-vscode

Interfaces are cool!

Tue, 31 Oct 2017 20:00:45 +0000

A while back I blogged about learning about interfaces as I didn’t really understand the value of them. I do now.

I created an application that used interfaces so I could learn how it worked. I created a Logger Interface and created multiple classes that implemented that interface so I could swap out the different implementations easily. I created a SQL Logger and a File Logger and my code could be written and be completely unaware of which implementation it was using.

This application uses SQL Azure and so I have a monthly bill to pay. Wouldn’t it be cool if I could reduce this bill? How about using the cheaper table storage instead?

Easy!

Create a new class that implements my interface and all I need to do is write the three methods defined in my interface and I can swap from SQL Azure to table storage.

Another benefit to interfaces is testing. Say I have an interface called inotification for sending notifications, I can have several implementations of this email, twitter, slack etc

None of these implementations should be used in unit tests, as you don’t want a tweet being sent every time you run your tests. Why not create an implementation that simply returns something for each method call and doesn’t actually do anything. I can then run my tests with my fake implementation which tests my code logic but not the implementation I have chosen (this can be tested later on with integration tests or user testing if required).

This is pretty much all I have to say about interfaces. I just like how I can swap different implementations.

It does take a bit of work to get the interface setup. I found that when writing the second implementation the interface would need to change slightly, mostly as it was badly designed to begin with. I think for beginners there may be some value to writing multiple implementations of an interface so you can be sure your interface is good, however I am sure with experience this will not be required.

Creating DNS records programmatically

Mon, 16 Oct 2017 20:00:45 +0000

DNS is the backbone of the internet. It converts domain names into IP addresses. But how can we do clever things with it?

Making DNS changes has always been a pain. You have to remember what company you registered your domain name with, then locate the login details for their website, login and fill in some web form and wait for the changes to replicate across the internet.

There are various websites that assign you a custom subdomain name like name.domain.com, obviously they must do this programmatically, they don’t have a guy following the above process.

Lets see how we could do this.

My first thought was to make use of the DNSimple API. https://developer.dnsimple.com/v2/ The API allows all sort of DNS changes to be made, however this is a paid service so let’s look at other options.

Azure now has a DNS section. You can go to the portal, add a DNS zone, and then add as many A, CNAME and MX records as you need. This is of course no better than doing it manually.

Azure offers a SDK and an example Visual Studio project. Lets look at how it works.

First you need to setup some credentials to use your Azure subscription. The easiest way to do this is with powershell.

Login–AzureRmAccount
$sp = New-AzureRmADServicePrincipal –DisplayName exampleapp –Password "{provide-password}"
Sleep 20
New-AzureRmRoleAssignment –RoleDefinitionName Contributor –ServicePrincipalName $sp.ApplicationId

This will create a AD Service Principal which has access to your Azure subscription.

You need to get the following IDs from the Azure portal.

subscriptionId for your subscription
tenantId or the Azure AD Directory ID
applicationId of the service principal created above
password you entered into the powershell script above

var serviceCreds = await ApplicationTokenProvider.LoginSilentAsync(tenantId, clientId, secret);
var dnsClient = new DnsManagementClient(serviceCreds);
dnsClient.SubscriptionId = subscriptionId;

var recordSetParams = new RecordSet();
recordSetParams.TTL = 3600;
recordSetParams.ARecords = new List<ARecord>();
recordSetParams.ARecords.Add(new ARecord("1.2.3.4"));
var recordSet = await dnsClient.RecordSets.CreateOrUpdateAsync(resourceGroupName, "funkysi1701.com", "test", RecordType.A, recordSetParams);

The above code connects to Azure and creates an A record for test.funkysi1701.com that points at the ip address 1.2.3.4.

Other DNS records can be created in a similar way.

The above example creates a new RecordSet but you can use the following to delete or get existing records.

var recordSet = dnsClient.RecordSets.Get(resourceGroupName, "funkysi1701.com", "test", RecordType.A);

dnsClient.RecordSets.Delete(resourceGroupName, "funkysi1701.com", "test", RecordType.A);

Now I have found out that I can write a console app to edit my DNS records I need to change the nameservers for all my domains so I can take advantage of this new feature.

Discovery – My thoughts so far

Tue, 10 Oct 2017 20:00:45 +0000

I tried to resist but I am going to have to write about the new Star Trek series Discovery. Warning this post is going to include SPOILERS. If you read on you have been warned.

I watched the first two episodes of the brand new Star Trek TV show. As I am an international viewer I used Netflix, if you are from the US you need to use CBS All Access.

I have heard a lot of complaining that the show is not free to watch. CBS is making Star Trek Discovery to make money, if they don’t make money they will stop making it. Its as simple as that.

Netflix costs £5.99 per month and CBS All Access costs $5.99 per month. I don’t believe this is a lot of money. The 5.99 above allows you to watch any of the 726 Star Trek episodes at any time of day whenever you want as many times as you want, plus you have access to all the other movies or TV shows available. I don’t think this is too much to ask.

Back to Discovery. I loved it! It felt like Star Trek. I had feared it might only share the name and would be an action filled TV show that had little in common with what he had seen before. I was wrong this is definitely a show that can proudly call itself Star Trek.

The Klingons

When it was first announced that the Klingons would feature in the new show I was a bit “meh”. We had done lots with the Klingons before and they had never been my favourite alien race.

The look of the Klingons was also going to be changed, I have to admit this didn’t bother me. Klingons have had their look updated before. In 1966 Klingons had dark faces and smooth foreheads, in 1979 the forehead ridges were added as the first Star Trek movie was made, and now Discovery has removed all hair from Klingons. I can explain it away as productional changes, I don’t need an on screen explanation like we had on Enterprise (or even DS9).

What I have seen so far is a menacing alien race that fits with what we have seen before with lots of references to honor, Kahless and speaking Klingon. I must admit reading all the subtitles is getting a bit tiresome, but that is a minor issue.

Conflict

Historically Star Trek has not featured conflict between the main starfleet characters due to the idea that humanity has evolved beyond this. DS9 got round this by having Odo and Kira who are not starfleet characters so can have a little bit of conflict. Discovery has completely abandoned this idea.

In the pilot episode, the Michael Burnham commits mutiny on her captain, even attacking her with a vulcan nerve pinch. In the third episode we finally meet the discovery crew. Gone are the TNG days where the ships crew are like a family, I am not sure I can think of a single character that would call another character “friend”. We are in a time of war so this would be expected, however I do hope we see deepening friendships form between characters.

I am OK with the change to feature more conflict. I must admit the darkness of the third episode, did stop to make me think a bit, however by the fourth episode I was won over.

Cast

The story is concentrated around Michael Burnham, so I do worry that other characters won’t get a look in. However from what I have seen Michael Burnham is a great character. She is a strong female character, with a intriguing back story relating to Sarek and possibly also Spock.

Saru the alien character on the show, played by the very tall Doug Jones is great. From the trailers he was all about sensing death, but there is far more to him than that. I am looking forward to learn more about him and his threat ganglia.

As the show has gone on we have started to learn about the other characters like Captain Lorca, Tilly and Paul Stamets.

Title Sequence

I don’t like the opening title sequence, it feels very cheap like a draft version which hasn’t been finished yet. I was looking forward to a title sequence that would show off the USS Discovery, maybe like TNG where it warps around our own solar system. What we see is some of the tech that features in Star Trek which is nice but I want more. The theme tune while great that it features parts from the classic theme doesn’t stand on its own. I can hum all the other TV shows themes, and now after 4 episodes would struggle to do that.

However I expect that after a while I will grow to like this more.

Overall

I like Discovery and I will keep watching it. I want to find out what happens to the characters. There are no annoying characters like other shows have had. There is character development, none of the characters are going to finish where they started. I want more friendship between characters, but I expect that will come, I want more exploring and doing Star Trek stuff, but most of all I want more episodes, roll on next weeks show.

Other things I noticed

Sound effects on the bridge are awesome, it makes me feel at home
The USS Discovery doesn’t feature in the first two episodes despite being the title of the show.
Others have noted that Captain Georgiou has books on her shelf which feature classic episode titles

Star Trek is back with Discovery

Fri, 22 Sep 2017 20:00:45 +0000

In November 2015 it was announced that a new Star Trek series was going to be launched. It has been a long wait with multiple delays but Star Trek Discovery is finally here.

Star Trek Discovery launches in the US on 24th September and in the UK on 25th September.

Lets have a look back how we got here.

Star Trek was created by Gene Roddenberry in the 1960s. He imagined a show where we would explore the galaxy while also exploring the human condition. He described it as a wagon train to the stars. However it was cancelled after 79 episodes and three seasons due to budget cuts and low viewers.

After cancellation its popularity boomed in reruns. In the mid 1970s an animated series was created. But demand for Star Trek continued until a new Star Trek show was commissioned called Phase Two.

In 1977 Star Wars came out, and the phase two idea was cancelled and turned into a movie. In 1979 we got Star Trek The Motion Picture. It went massively over budget and failed to really capture the Star Trek spirit.

A much cheaper sequel was created in 1982 Star Trek II: The Wrath of Khan with Roddenberry taking more of a back seat. A film widely considered to be one of the best trek films. Four more feature films starring the original cast were made.

However in 1987 Gene Roddenberry teamed up with Rick Berman and The Next Generation was created with a brand new cast. The series ran for seven seasons set roughly a century after the adventures of Kirk and Spock.

Popularity of Star Trek soured and this led Rick Berman to help create the spin off series Deep Space 9, Voyager and lastly Enterprise along with four feature films continuing the adventures of the TNG cast. When Enterprise was cancelled in 2005, Star Trek appeared to be dead after 18 years on TV.

However in 2009 movie creator JJ Abrams recast the classic crew for the big screen. The film was big on action, but took place in an alternative timeline allowing for familiar characters to have new adventures without interfering with events depicted in the TV shows.

Two sequels to this film have been produced and are bringing Star Trek back into popularity. This has given rise to Discovery. Star Trek is returning to the TV screen and it is going to be great.

Azure Friday

Mon, 18 Sep 2017 20:00:45 +0000

I think Azure is great, but there is loads to it so I can never know about all of its features. There is a video series hosted by Scott Hanselman called Azure Fridays which I have started to watch in an effort to keep more up to date about some of its cool features.

I watched this video recently and it is all about application insights and new ways you can debug your web applications by creating snapshots. I am a big fan of application insights so adding extra ways to debug my apps is a big win for me. Once I get this feature working in my code I will no doubt blog about it.

Azure Functions

Tue, 12 Sep 2017 20:00:45 +0000

I recently blogged about using Azure Web Jobs , Azure Function is another way of doing the same thing, lets look at how they work.

(Sorry its been a while since I blogged but I suspect an erratic schedule will continue for the next few months.)

To create an Azure Function go to the Azure portal and click add new and search for “Function App”

Give your app a name and select the usual resource group and location settings.

Now when you open Function Apps you should see your new app.

I want my Function App to run on a schedule so I clicked the + next to functions and selected TimerTrigger. I am a c# programmer so I selected this option as well.

Give your function a name and specify using the usual cron notation how often it should run. I want mine to run at 9.30pm each night so use 0 30 21 * * *

Now comes the code bit. By default you get a window with the following code in it

using System;

public static void Run(TimerInfo myTimer, TraceWriter log)
{
 log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
}

It is entirely up to you what you get your function to do. In my case I just wanted to call a URL on a schedule so I created some code that used httpclient.


using System;
using System.Net.Http;

public static async Task Run(TimerInfo myTimer, TraceWriter log)
{
 log.Info($"Buffer 0 function executed at: {DateTime.Now}");
 HttpClient client = new HttpClient();
 var result = await client.GetAsync("URL");
 string resultContent = await result.Content.ReadAsStringAsync();
 log.Info(resultContent);
}

Once you have created your app and it has run you can use the monitor section to view success and failures.

There is loads more you can do with Azure Function but this is a good place to start.

Pwned Pass – Available from the Play Store

Mon, 14 Aug 2017 20:00:45 +0000

Pwned Pass is now available from the Google Play Store .

Pwned Pass is a simple android app that allows you to type in a password and tells you if it has been used in a data breach.

Troy Hunt of Have I Been Pwned? recently added a new API to his website which allows you to search his extensive database of pwned passwords, 306 million of them. I have simply created a Android frontend to this API.

The API itself takes a SHA1 hash of the password and either returns a HTTP 200 if the password is found or a HTTP 404 if the password does not exist in the HIBP database. For more details of how Troy Hunt created this check out his blog post .

My app simply generates a SHA1 hash of anything that is typed in and then passes this to Troy Hunt’s API. I then get the HTTP return code so I know if the password exists or not.

It should be noted that: Do not send any password you actively use to a third-party service – even this one! I don’t log anything that you type into my app and all I am then doing is passing a SHA1 hash over SSL to HIBP. However you shouldn’t trust my word alone.

The app itself is written in Visual Studio with Xamarin Forms in a similar fashion to the app I talked about last week .

As I am using Xamarin Forms there is the potential that I may develop iPhone or UWP versions of this code in the future. With that in mind I have made use of interfaces for the android specific parts of the code.

I also make use of the modernhttpclient nuget package due to problems I encountered with httpclient and SSL. This is due to limitations of what libraries are available in mono and what has been implemented, I suspect there are better ways to solve this but that is all part of the fun.

Please do have a look at Pwned Pass and let me know what you think. Especially if it doesn’t work or throws errors. I would like to spend time making this app as good as I can make it.

Android App Development and the Visual Studio Mobile Centre

Mon, 07 Aug 2017 20:00:45 +0000

For the past week or so I have been playing around with Xamarin and creating an android app.

Well I now have an app in the Google Play Store. Check out https://play.google.com/store/apps/dev?id=6148298088834956775 .

Before you rush and download the app I must warn you that it doesn’t do much yet. It displays some content that is on my website and there are a few links to allow sharing of content. I have some ideas to display content from my blog and allow sharing. I also have some other ideas for apps that might actually be useful to people that are not me. If you have ideas or feature requests do let me know.

Ok how did I go about creating this app and getting it in the app store?

Xamarin is now part of Visual Studio so step one is install all the Xamarin features to Visual Studio and build an app.

Next I wanted to monitor my app. Now I know Application Insights doesn’t support apps so what tools are out there? HockeyApp is something I had heard of but they are in the process of being replaced with Visual Studio Mobile Centre .

It was relatively easy to hook up my app to Visual Studio Mobile Centre. First install the required nuget packages. Then add using statements and the following line to your MainActivity.cs file (these instructions are available on the Mobile Centre)

using Microsoft.Azure.Mobile;
using Microsoft.Azure.Mobile.Analytics;
using Microsoft.Azure.Mobile.Crashes;
using Microsoft.Azure.Mobile.Distribute;
using Microsoft.Azure.Mobile.Push;

MobileCenter.Start("[Unique ID]",typeof(Analytics), typeof(Crashes), typeof(Distribute), typeof(Push));

Now you can connect the Mobile Centre to your source code (VSTS in my case) and get it to run a build for every commit.

One complexity of the build is that you need to supply a keystore file (basically a certificate to digitally sign your app). I found the best way to do this was to use Visual Studio to create the file.

In VS2017 there is a option called Archive Manager under the tools menu. In here click the distribute button and select Ad-hoc. In the signing identity section you can create a keystore file. Enter a few details and a keystore file will be created in AppData\Local\Xamarin\Mono for Android\Keystore[keystore name][keystore name.keystore]

Once you have added the keystore file to your build you can enable the distribute option. Now you will get an email after every build with a link to install your app.

Every time your app crashes the details will be logged in the crashes section for you to explore and fix the issues.

The Analytics section allows you to explore how your app is being used. You can also add Analytics.TrackEvent(“Feature X”) to measure the usage of different features.

There are more things you can do which I will explore more at another time along with how to get your app into the Google Play Store.

Creating your own nuget packages with VSTS

Mon, 31 Jul 2017 20:00:45 +0000

For a while I have found myself writing the same bits of code for different web projects. This annoys me as it goes against the DRY principle (don’t repeat yourself).

One possible solution is to write your own nuget packages. You can then add this piece of code to any project you work on.

nuget.org is the public nuget feed where any developer can download nuget packages. You could publish your nuget package here, but your might want to restrict access so better to create a private nuget feed.

Lets look at how we create a nuget feed in Visual Studio. First thing you need to do is install the Package Management extension to Visual Studio Team Services (its free for less than 5 users), this will add a packages section under the build menu.

Before you can start using this new feature you need to add a Package Management License in the users hub.

Once that is done you can create a feed. You need to give your feed a name, decide if only members of the current project or everyone in your account should have access to read and contribute to.

Now you have a feed you could use the nuget package command to create a nupkg file and then nuget push command to add it to your feed. A better way is to get Visual Studio Team Services to do all the hard work.

Create a new project in Visual Studio Team Service to house your nuget package. In the build section add an empty build definition. Choose a build agent, I am using the Hosted VS2017. Then add the following steps nuget restore, Visual Studio Build, nuget pack and nuget push.

nuget restore – this step is only needed if your code depends on other packages. If it depends on other packages that are only in your feed you must specify your feed in the feeds and authentication section.
Visual Studio Build – this builds your code like you would in Visual Studio. The only config I made to this step was to specify Release in configuration.
nuget pack – this creates the nupkg file from your built project. In configuration to package specify the same as you specified in the previous step (in my case Release)
nuget push – this publishes to your feed, so of course you need to specify your feed.

One last thing to configure is to enable the continuous integration option in triggers. This means whenever you push code all these steps will run and you have a new version of your nuget package.

In Visual Studio you need to create a *.nuspec file, this contains all the meta data about your nuget package. Let look at an example.

<?xml version="1.0"?>
<package >
 <metadata>
 <id>$id$</id>
 <version>1.0.2</version>
 <title>Nuget</title>
 <authors>Simon Foster</authors>
 <owners>Simon Foster</owners>
 <requireLicenseAcceptance>false</requireLicenseAcceptance>
 <description>An example of a nuget package.</description>
 <releaseNotes>Release Notes</releaseNotes>
 <copyright>Copyright 2017</copyright>
 <projectUrl>https://[yourVSaccount].visualstudio.com/nuget/</projectUrl>
 </metadata>
</package>

One last thing to mention is version numbers. You can either change the version number in your *.nuspec file everytime you push changes. This will create stable packages like 1.0, 1.1, 1.2 etc

However you can use the automatic version number setting in the nuget pack build step. However I have found this only ever creates pre-release packages and I haven’t found a way to upgrade a package from pre-release to stable.

This is a really neat way to reuse your code in multiple projects. I have only been looking at this for a few days and I have already extracted code to do with emails, creating excel downloads and database related methods. I suspect that doing this will also have a side benefit of forcing me to create code with fewer dependencies so more code can be turned into a nuget package.

Async and Await

Mon, 24 Jul 2017 20:00:45 +0000

For a while the Async and Await commands in c# have confused me.

Like most things the best way to learn about something is to use it in a real world example. I am currently adding an email alert feature to a website. This is an ideal example of something that would benefit from Asynchronous programming. There is no need for the webpage to wait to send 1000s of emails, lets just send a call to get started and allow the browser to carry on as normal.

This is my first try at using async and await so feel free to suggest best practises in the comments.

Lets start with a Send method in my EmailController.

public ActionResult Send(int id, int pageId, int userID)
{
 if (!Authorize.checkPageIsAuthorised(userID, (Authorize.PageIds)pageId))
 {
   return Redirect("/login");
 }
 else
 {
   Task<string> t = SendNotifications(id,userID);
   return Redirect(Request.UrlReferrer.ToString());
 }
}

This simply checks to see if you have permission to the page. If not redirects to the login page otherwise it makes a method call and redirects back to the page it came from.

Lets have a look at that method call in more detail.

Task t = SendNotification(id, userid);

SendNotification doesn’t return a normal string it returns a Task, so lets look at how we are creating this.

public async Task<string> SendNotifications(int id,string type,int userid)
{
 //logic ommitted
 await ef.SendEmail(model, emailHtmlBody);
 return "OK";
}

The return type is set to Task but it has the aysnc keyword appended to it. It also makes a call with the await keyword.

public async Task SendEmail(EmailModel model,string emailHtmlBody)
{
 //logic removed
 await smtp.SendMailAsync(message);
}

So that is it. My first bit of code that uses Async and Await. My controller calls a method asynchronously which then calls another method asynchronously which sends emails asynchronously.

Async – This enables the Await keyword to be used in the method

Await – This is where things get asynchronous. The await keyword allows the code to wait asynchronously for the long running code to complete.

How good is your SSL?

Mon, 17 Jul 2017 20:00:45 +0000

I recently watched Troy Hunt’s What Every Developer Must Know about HTTPS course on Pluralsight. Its very good and really make you think about SSL certificates and how to correctly implement them.

One thing in particular Troy mentions is the website SSL Labs . This website allows you to test a websites implementation of SSL. A grade of A to F is assigned with A being the best and F being the worst.

Troy Hunt has a blog post where he discusses how Australian Banks score. Lets look at a few UK banks.

Bank	SSL Certificate	Grade	Home Page Under SSL
HSBC	www.hsbc.co.uk	B	Y
Nationwide	onlinebanking.nationwide.co.uk	C	N
NatWest	www.nwolb.com	C	N
Barclaycard	www.barclaycard.co.uk	A-	Y
Barclays	bank.barclays.co.uk	A-	N
Lloyds Bank	www.lloydsbank.com	A	N
Royal Bank of Scotland	www.rbsdigital.com	C	N
Standard Chartered	www.sc.com	C	Y
Virgin Money	uk.virginmoney.com	A+	Y
Santander	retail.santander.co.uk	A-	N

On the whole the ratings are all quite good with all being in the range A-C. However I have also indicated if they have SSL on the home page. Only 4 out 10 website listed above have the home page load under SSL.

Why does this matter as long as the login is under SSL? Any page that loads over http is potentially at risk from a man in the middle attack. A fake malicious home page could contain links to any page and trick users into entering personal information.

If you want to test a bank or other website not listed here. Go to https://www.ssllabs.com/ssltest/index.html and type the address that is on the SSL certificate in to the search. The good news is that this site scores a A.

Troy mentions that there is rapid growth in the adoption of SSL, there is also rapid growth in improving ratings. One of these banks went from a C to an A during the course of writing this blog.

Writing your first test

Mon, 10 Jul 2017 20:00:45 +0000

Whenever I write a new test I have to think how best to do it. Hopefully I can summarise a few tips here to help get started.

Arrange Act Assert

The first thing I think about when writing a test is Arrange, Act, Assert. Arrange, Act, Assert is a pattern for writing the tests.

Arrange – This gets things in order ready to execute the test.

Act – This executes the method you want to test.

Assert – This compares the value produced in the Act step with a known value typically with a method similar to the following

Assert.AreEqual(expected value, actual value)

Say for example you wanted to test a method called ReturnsTrue() which does nothing but returns a value of true. This method is in a class called ReturnsTrueClass

The Arrange step in this example would be.

ReturnsTrueClass t = new ReturnsTrueClass();

The Act step in this example would be.

var result = t.ReturnTrue();

The Assert step in this example would be.

Assert.AreEqual(true, result);

This is a stupidly simple example but hopefully you get the idea of how you can build all your tests with these three steps.

Recently I saw a tweet complaining that someone has mixed up expected and actual in the Assert statement.

There is a minor but special hell reserved for those who mix up the expected and actual parameters in Assert.Equals
— Keith Williams (@zogface) July 5, 2017

At first glance this probably isn’t the worst mistake to make as if your tests are all passing actual and expected are the same.

However tests will fail, that is the whole point of them, you can then fix bits of code. If you have mixed up actual and expected it adds extra time to debugging and figuring out what values are produced from your code and what you are expecting it to produce. It may be your test uses a mocking framework and somewhere in there, there is an issue, with mixed up expected/actual you may assume a problem in your code rather than the test.

Also, how do you make such an error? When I type Assert.AreEquals() in Visual Studio, Visual Studio tells me what each parameter does, it takes a matter of seconds to do this, just by hovering over the code.

One last tip to say about tests. Write your tests to test the behaviour of your application.

Uploading Files to AWS

Mon, 03 Jul 2017 20:00:45 +0000

I am a fan of Azure but today I have been looking at AWS. Specifically how to upload and download files.

AWS S3 stores files in Buckets. I already had an AWS S3 account setup with a Bucket. I am going to assume you have got a bucket setup and concentrate on the code to get files in and out.

First step is to use nuget to install the AWS packages. In nuget the packages you want are called AWSSDK.Core and AWSSDK.S3.

The using statements you want to use are called Amazon.S3 and Amazon.S3.Transfer, not sure why this doesn’t match nuget, this difference caught me out a couple of times.

Now to the code that uploads files

AmazonS3Client AWSclient = new AmazonS3Client(accessKeyID, secretAccessKeyID, Amazon.RegionEndpoint.EUWest1);  
TransferUtility fileTransferUtility = new TransferUtility(AWSclient);  
using (FileStream streamWriter = new FileStream(path, FileMode.Open))  
{  
  TransferUtilityUploadRequest fileTransferUtilityRequest = new TransferUtilityUploadRequest  
  {  
    BucketName = "flawlessimages",  
    InputStream = streamWriter,  
    Key = fileName  
  };  
  fileTransferUtility.Upload(fileTransferUtilityRequest);  
}

Lets break it down and look at what it does.

AmazonS3Client AWSclient = new AmazonS3Client(accessKeyID, secretAccessKeyID, Amazon.RegionEndpoint.EUWest1);

This creates an instance of AmazonS3Client, we are passing the Access Key and Secret Access Key both of which can be found from your Amazon S3 account My Security Credentials section. Amazon.RegionEndpoint.EUWest1 specifies the amazon data centres that your bucket is located in.

TransferUtility fileTransferUtility = new TransferUtility(AWSclient);

This creates an instance of TransfterUtility using the AmazonS3Client instance we created in the previous step.

using (FileStream streamWriter = new FileStream(path, FileMode.Open))  
{

This opens up a filestream from a files path and specifies that the file should be opened.

TransferUtilityUploadRequest fileTransferUtilityRequest = new TransferUtilityUploadRequest  
{  
  BucketName = "flawlessimages",  
  InputStream = streamWriter,  
  Key = fileName  
};  
fileTransferUtility.Upload(fileTransferUtilityRequest);

This last step specifies which bucket to upload to, what input stream to upload and the Key to use. Key is just AWS way of referring to files, more commonly referred to as the filename.

This is all you need to do to upload a file to your Bucket. The file will be located at https://s3-eu-west-1.amazonaws.com/[bucketname]/[filename] , however by default it will not be downloadable until you set Read permission to everyone, once you do that anyone who has the link will be able to download your file.

This is the same permission level as any file you have on your webserver, however AWS has a better way.

using (s3Client = new AmazonS3Client(accessKeyID, secretAccessKeyID, Amazon.RegionEndpoint.USEast1))  
{  
  GetPreSignedUrlRequest request1 = new GetPreSignedUrlRequest  
  {  
    BucketName = bucketName,  
    Key = filename,  
    Expires = DateTime.Now.AddMinutes(5)  
  };  
  urlString = s3Client.GetPreSignedURL(request1);  
}

Here we are generating a url to download the file, but we are specifying that it is only valid for 5 minutes. This means that if you share the url it will only work for 5 minutes, after that AWS will give an access denied message.

This is much better security than you have on a typical web server, and easy to implement, every time a user clicks on a download link you generate a new presigned url and send the download to the browser, as long as this process doesn’t take longer than 5 minutes the user will never know.

Using Azure WebJobs to Automate Stuff

Mon, 26 Jun 2017 20:00:45 +0000

I keep hearing about Azure WebJobs but I have never used them. Time to change this.

WebJobs are a feature of Azure App Service that can run a script at a specific time. In my case I would like to hit a specific url of my website at the same time every day.

To the right you can see an example of the WebJobs form on the Azure portal that you need to fill in.

You need to supply a name for your webjob.

You need to upload the script that will run in my case I used a powershell script. My script consisted of which basically just loads the url specified.

$progressPreference = "silentlyContinue";
$result = Invoke-WebRequest -Uri ("https://www.google.com") -Method Get -UseBasicParsing;

Type refers to if your job will be triggered or run continuously, I want it to be triggered.

Triggers refers to if you want it to be scheduled or manual, something that you can run on an ad hoc basis. I of course want scheduled.

If you are familiar with the linux CRON then the next box will make sense to you for everyone else I will try and make sense of it. The box consists of 6 numbers which can either have a value or a *. The numbers correspond to the following {second} {minute} {hour} {day} {month} {day of the week}.

A hourly job would be expressed as 0 0 * * * *, ie every day of week, every month, every day, every hour and only when minute and second equals zero. For more help with this check out the MSDN docs about it. I want to use 0 30 21 * * * to run daily at 9.30pm.

That’s it everything setup, now time to wait and see if it works.

Oh no!

It failed to run at the specified time.

The reason for this is the scheduler requires the feature Always On to be turned on which is not available in the free App Service. Before you reach for your wallets, I found a solution on this blog post that allows them to run on the free tier.

The thinking behind this solution is you need to keep the website alive throughout the day so Tom has created a script that does this. His script can be found on his blog or on his github page .

Set this script up to run every 5 minutes (0 */5 * * * *) like the example above.

The nextthing you need to do is create a Custom connection string in the Application Settings blade called SecretThing. Tom’s script references this to access the website and keep it alive. The password you need to put in SecretThing can be found in you publish profile (downloaded from the Overview blade in the Azure portal). For more details and a better explanation check out Tom’s blog .

One last thing to mention about WebJobs is that you can see details about when they have run at https://[YourWebAppName].scm.azurewebsites.net/azurejobs/#/jobs and this can be a great place to help debug your scripts.

Fiddler and APIs

Mon, 19 Jun 2017 20:00:45 +0000

A while ago I blogged about promoting my blog with Buffer. At the time I made use of the nuget package BufferAPI but lets look at some improvements I can make.

The BufferAPI package worked great from my console app, but when I tried to use it from a Controller in an MVC app I never got it to work. Lets look at the API docs and see if I can rewrite it.

There are two main types of API calls GET which gets data from the server and POST which posts data to the server. These come from the types of HTTP requests.

I quickly figured out how to use the GET API call to authenticate using https://api.bufferapp.com/1/profiles.json?access_token=XXXX

However POST was defeating me. That was until I remembered Fiddler .

I had heard Troy Hunt (and others) talk of using Fiddler to examine what data is being passed among websites. Troy uses it to do a man in the middle test to see what information can be stolen.

It is really easy to setup, install Fiddler, click yes to a few security warnings and you can see what information is being passed from your code to remote APIs.

Once I had Fiddler installed I could compare what information is being passed between a successful API call using the BufferAPI nuget package and an unsuccessful API call using my code.

Fiddler also showed that passing my authentication token in a POST request is much better. Despite both GET and POST being encrypted when using HTTPS, anything at either end that logs URLs will have a log of your username and password.

If you have not tried Fiddler, give it a try especially if you are doing things with API calls.

How do I add Power BI data to a webpage?

Mon, 12 Jun 2017 20:00:45 +0000

Last week I talked about Power BI , what it is and some of the different services you can use with it. This week lets add some of that data to a simple web page.

For this example I am going to add the google analytics data from this website to this page.

First login to your Power BI https://app.powerbi.com/

Click the get data link at the bottom left.

Click My Organisation to bring up the app search box.

Click the Apps tab and search for “google” in the search box, you should then see Google Analytics, click into this and then click the get it now button.

Log into your google account. If you have multiple google accounts I found it worked best to sign out of all of them or run this in an incognito window.

Once you are signed in you should see a list of the different google analytics data you have, select the one you want to use and click import.

Power BI will then go away and start loading the data.

Once loaded go to Reports and select the Google Analytics that has been loaded. If you have more than one, it is a good idea to rename each one eg Corporate Site Google Analytics, Blog Google Analytics so you won’t get mixed up.

In the file menu select Publish to web and agree that you are OK for this to be made public.

You will then be given a piece of HTML code that starts with <iframe copy this onto your web page. Reload your webpage and you should see something similar to mine below.

It should be noted that while you can but the webpage containing the iframe behind a login page, the data could still be accessed if you knew the url contained within the iframe, this is why the link can be emailed and continue to work.

What the heck is Business Intelligence?

Mon, 05 Jun 2017 20:00:45 +0000

The term Business Intelligence (BI) refers to technologies, applications and practices for the collection, integration, analysis, and presentation of business information. The purpose of Business Intelligence is to support better business decision making. Essentially, Business Intelligence systems are data-driven Decision Support Systems (DSS). Business Intelligence is sometimes used interchangeably with briefing books, report and query tools and executive information systems.

What is Power BI?

Power BI is a business analytics service provided by Microsoft. It provides interactive visualisations with self-service business intelligence capabilities, where end users can create reports and dashboards by themselves, without having to depend on any information technology staff or database administrator.

Microsoft describe it as follows: Power BI is a suite of business analytics tools that deliver insights throughout your organisation. Connect to hundreds of data sources, simplify data prep, and drive ad hoc analysis. Produce beautiful reports, then publish them for your organisation to consume on the web and across mobile devices. Everyone can create personalised dashboards with a unique, 360-degree view of their business. And scale across the enterprise, with governance and security built-in.

What can you do with it?

One of the first thing you can do with Power Bi is connect some of the apps that you use.

To sign up to use Power BI go to https://powerbi.microsoft.com/en-us/

Once you are signed in click the get data link in the bottom left corner and browse through the apps you can use.

If your website uses google analytics you can connect it up to Power BI and see some cool analytics. The below map was generated from some google analytics data, you can see that this website has had users across the globe, with UK and USA getting a lot of hits.

Another thing you can do is connect Azure Application Insights. This provides some similar data to google analytics about who is visiting your site.

If your code is hosted in Visual Studio Team Services you can get some cool analytics from your commit history.

There is a huge long list of apps you can connect to power BI so you can start getting all sorts of analysis very easily. I am hopefully going to talk another time about getting custom data into Power BI.

Have I Been Pwned?

Mon, 29 May 2017 20:00:45 +0000

Have you heard of the website https://haveibeenpwned.com ?

Well you should have. Have I Been Pwned is a website created by security expert Troy Hunt that keeps track of data breaches and allows you to search and find ones that affect you. As I write this Troy Hunt has tracked 3,752,984,562 pwned accounts from 216 pwned websites.

What does “pwned” mean?

pwn means to compromise or control, specifically another computer (server or PC), website, gateway device, or application. (as defined on wikipedia)

Originally, pwn and its variants were pronounced /ˈoʊn/ in the same way as the verb own, the tail of the p being “silent”.

In terms of this site Troy Hunt defines it as:

A “breach” is an incident where a hacker illegally obtains data from a vulnerable system, usually by exploiting weaknesses in the software. All the data in the site comes from website breaches which have been made publicly available.

What can I do?

As an individual you can search for your email address in Have I Been Pwned, I am in 7 data breaches.

As a person responsible for an email domain you can search and find which of your users are in a data breach.

You can also sign up to get notified of any future data breaches you might get caught up in.

If you are in a data breach change your password. If you use the same password across multiple sites or services change them as well. Consider using a password manager so you can have unique complex passwords for every services you use and not have to worry about forgetting them.

If you look after a website or service then follow Troy Hunt. Think about security, is your site vulnerable to SQL injection, do you store passwords with reversible encryption?

How worried should I be?

Data breaches are happening more and more often. Its not showing any sign of slowing down, in fact I expect there to be lots more in the years to come. However there are things you can do to mitigate the damage of being in a breach.

Use a password manager
Don’t share passwords between sites
Regularly change your passwords
Think of passwords as pass phrases and include spaces between the different words

Consider what information a company has about you. How worried would you be if this became public knowledge? Consider if you want this online, weigh up the benefits etc

Star Trek Discovery – thoughts on new trailer

Mon, 22 May 2017 20:00:45 +0000

I am really excited a new Star Trek Discovery trailer has just been released, if you haven’t seen it yet, watch it now.

The rumour mill has been going crazy about Star Trek Discovery but now we have a trailer with actual footage to analyse. In my opinion all the concerns about changes in the production team have gone out the window, this looks to be a polished Star Trek show that can sit alongside our favourites.

The Ship

When images first surfaced of the ship I wasn’t keen, however all the shots of the ship look great. I assume this must be the USS Discovery, although it is hard to tell anything from a two minute advert.

The Bridge

The Bridge looks recognisable as a Star Trek bridge, but also looks fresh and new. I think it feels very JJ Verse. I like it.

The Uniforms

The uniforms look like a mid step between what we saw on Enterprise and what we know as the Classic Trek look. I can’t see any Blue, Red, Gold to signify departments, but it looked like Gold and Silver could well signify something. Really hard to tell at this stage, but I like what I can see.

Characters

The show feels like it is going to be gritty and real. With characters you can really get behind. I liked what I saw of the captain and lead character. The alien character intrigued me, hopefully his line about death will be explored. It feels like there is a strong link between Vulcans and Sarek with the shows lead (played by Sonequa Martin-Green) Not sure how I feel about this yet, depends on what they do with it. I don’t want them to just redo what they have done with Spock, but again lets wait and see.

Klingons

OK I am not sure I like the strong emphasis on the Klingons. I think we have spent a lot of time learning about Klingons, so when I heard the casting news about Klingons I was not excited. The Klingons also have a new look, which doesn’t bother me that much as the Klingons have changed their look before. But lets wait and see what happens.

Star Trek is back

After over a decade with no Star Trek on TV, it is finally coming back. I am beyond excited about this. There are lots of things I want from this new show but this advert has definitely rekindled my excitement and hope that we are going to get something good.

Windows Update – don’t turn it off!

Mon, 15 May 2017 20:00:45 +0000

No blog post this week. However why not check out Troy Hunts blog article about Windows Update .

I am probably fairly unique in that I like installing updates whether it be on my phone, my PC or for one of my favourite programs. One of my tasks when I used to work in a sysadmin role was ensuring all windows updates got installed on all servers and client machines. It was a never ending task, as soon as you got almost all updates done, it would be update Tuesday and a new set of updates would be released.

I can think of a couple of times an update got installed that caused a problem and needed to be rolled back. However I have no way of measuring how often it fixed a problem or improved the security of the OS or other software. If you compare the annoyance of waiting for updates to install against having a computer (or worse a server) be hacked or otherwise compromised I think it is fairly clear what you need to do.

As someone who works in IT I feel it is my duty to share the importance of keeping your devices updated. Don’t turn automatic updates off or disable it. It is there to help you and keep your computer safe and secure.

HighCharts

Mon, 08 May 2017 20:00:45 +0000

High Charts is a javascript library that allows graphs and charts to easily be added to web pages.

A chart like this can easily be added with a few lines of html and javascript.

<script src="https://code.highcharts.com/highcharts.js">
<script src="https://code.highcharts.com/modules/exporting.js">
<div id="container"></div>

This adds the highcharts library and a container for the chart to be displayed in.

Highcharts.chart('container', {

  title: {
    text: 'Solar Employment Growth by Sector, 2010-2016'
  },
  subtitle: {
    text: 'Source: thesolarfoundation.com'
  },
  yAxis: {
    title: {
      text: 'Number of Employees'
    }
  },
  legend: {
    layout: 'vertical',
    align: 'right',
    verticalAlign: 'middle'
  },
  plotOptions: {
    series: {
      pointStart: 2010
    }
  },
  series: [{
    name: 'Installation',
    data: [43934, 52503, 57177, 69658, 97031, 119931, 137133, 154175]
  }, {
    name: 'Manufacturing',
    data: [24916, 24064, 29742, 29851, 32490, 30282, 38121, 40434]
  }, {
    name: 'Sales & Distribution',
    data: [11744, 17722, 16005, 19771, 20185, 24377, 32147, 39387]
  }, {
    name: 'Project Development',
    data: [null, null, 7988, 12169, 15112, 22452, 34400, 34227]
  }, {
    name: 'Other',
    data: [12908, 5948, 8105, 11248, 8989, 11816, 18274, 18111]
  }]

});

This adds the data and sets up various options for the charts. More details about the different charts and options you can set can be found at https://www.highcharts.com

I am quite a fan of the different gauges that you can put on your site, as you can add a bit of animation and make the needles bounce around. https://www.highcharts.com/demo/gauge-solid

As most of the data I deal with lives in a SQL Server database, I have been spending quite a bit of time writing Stored Procedures and functions to mould the data so I can easily pass data from the database to javascript.

Highcharts are very flexible with lots of different options that allow you to display charts of almost any data you have, these charts are also exportable as PNG, JPEG, CSVs etc

Office Move

Mon, 01 May 2017 20:00:45 +0000

For the past few weeks my software developing has been taking a back seat as I planned and coordinated the IT requirements of an office move.

The company I work for has been working out of a converted barn, but as the company has grown we have outgrown the building and for quite a while it has been a real squeeze to get everyone in. We now have a shiny new offices with plenty of room for growth.

It was at the start of the year when I first started getting involved with the new office. At that time building work was about to begin. The office had concrete floors so in order to get network and power points in the floor, channels would have to be cut into the concrete for the floor boxes and cable runs.

At the old office our servers were in a cabinet in the corner of the room which meant they could be quite loud especially when the fans were going full pelt. Our American owners were supplying brand new IT equipment and we would have a dedicated server room.

Soon a weekly conference call was setup so we could coordinate with the IT people in America and the various different contractors that would help deliver our new office.

One thing I was particularly proud of was YDS. York Data Services (YDS) is an ISP I have worked with in the past at a previous job and I was able to continue my relationship with them and they were contacted and became our primary internet supplier.

Unfortunately most ISPs have to deal with BTOpenreach who have a bit of a monopoly on getting phone lines or leased lines installed. The initial estimate we were given was two weeks after we were due to move in. It was looking like we would move in and two weeks later we would get an internet connection. We were in the process of getting quotes for a temporary internet connection when BT contacted us and could get us connected up.

Soon afterwards a huge cabinet and two pallets full of equipment was delivered. Two days later a team was dispatched to rack everything up. On the rack we had the following equipment: 4 x UPS, 4 x Network Switches, 2 x Routers (for 2 x Leased Lines from different providers, the secondary connection has not been installed yet), 2 x Palo Alto Firewall devices, 2 x New Servers.

Just prior to our move we had a rack full of equipment but no patch cables, nothing connected up and I was beginning to get concerned that we would not be ready in time.

Thankfully a team was dispatched to help out over the weekend of the move, with the patch cables being delivered only hours beforehand. I worked closely with them and we worked late into the night. By the end of the Friday night our domain controller had been moved and its IP address updated. DHCP scopes had been setup for the new network. Once that was done we could get the WiFi points connected up (These had already been fitted to the ceiling).

The following day we moved our existing servers and updated their IP addresses and got everything patched up. Furniture started arriving so desks could start being setup and phones connected up. Everything was slotting into place, and for the first time our new office was starting to look like an office.

By the time it got to Monday the only problem was the phone number had not transferred as requested (something else to blame on BT) All staff could either connect via an Ethernet cable or connect to our brand new WiFi network and had access to all the IT services they had at the old office. Another minor issue was the NAS we used for backups had to be reset as it couldn’t communicate with the domain so no one could login, luckily this didn’t affect the data on it.

From my perspective the move was a massive success considering how complex and how many different people had been involved.

GitHub history

Mon, 24 Apr 2017 20:00:45 +0000

I created my GitHub account in August 2010, lets look at what I have done with it.

In 2010 I started by committing the code for an old php website I had created. Then in December I committed some other php sites.

Between 12 December 2010 and 29 September 2011 I must have created some automated process which is why there is a commit every day. Looking at the diff it appears to be related to tracking visitors to a site. I assume back then I hadn’t heard of the .gitignore file!

Nothing for a few years until 2014, when I started compiling a code samples collection, this is bits of code that I want to show off, I have added to this since 2015 so I really should go back to this as my skills have developed a bit since then.

In 2015 I really started to think of myself as a developer, I added repositories for Raspberry Pi, and a few C# ideas I had. I ended the year taking part in Advent of Code

What did I do last year? Well I started going to York Code Dojo and this meant lots of forks from their code examples and also my first pull request. I also did a fair bit of looking at other repositories trying to find a good open source project to contribute to, still not found one for me yet. Lastly I ended 2016 with a bit of Advent of Code.

Not sure how enthralling a blog post this is, but fascinating to see the different commit histories.

Automation of the Promotion of my Blog

Mon, 17 Apr 2017 20:00:45 +0000

I would like to automate the promotion of this blog. Currently to promote this blog on social media I use a few different services.

Buffer (buffer.com ) is a service that allows you to schedule updates to the main social media channels (Facebook, Twitter, LinkedIn and Google+)

IFTTT or If This Then That (ifttt.com ) is a service that allows you to connect different online services. You can send an email when a specific event occurs in your calendar for example.

I have been using a combination of these services to share to social media some of my past blog posts. I then add to my calendar details of my blog posts. Then I use IFTTT to add the event to Buffer, and then buffer tweets on a schedule.

This works great however it is a manual process to add my posts to my calendar. I have been using a spreadsheet to help me generate an ics calendar file which I import into my google calendar. There must be a better way of doing this.

I have written some code that reads the RSS feed of my blog and then shares that to Buffer using the Buffer API. The code I am creating is far from finished however I am trying to use the concepts of clean code to make it as flexible as possible.

I have an interface called ISocial which my buffer code implements, but it would be easy to add a class that implements the same interface but uses the twitter or facebook APIs. My code reads from a specific WordPress RSS feed, but it should be easy enough to adapt to read from a SQL database or any other data source.

I am currently unsure what kind of interface to use for this application. I could create a web page that controls the functionality, or maybe just a windows application, or maybe both of these are complicating things and all I need is just a console application that could be added as a scheduled task.

Blame

Mon, 10 Apr 2017 20:00:45 +0000

I recently came home from a busy day of work to my wife blaming me for allowing our 18 month old son to reach some paint and get it all over the carpet. She was concentrating on the fact that he could reach the paint not on the fact that she had brilliantly got the paint out of the carpet.

My instinct on being blamed is to deny it, or start throwing blame back at her. It is possible I put the paint in his reach, it is also possible it was someone else.

It is very easy to blame someone else. For example, looking through the git history to find out who changed a specific file is only a few clicks.

But is it ever productive to blame someone? Is it not better to focus our energy on fixing the issue at hand?

In the world of business it is easy to go from blaming other people until you have a blame culture. When you have a blame culture everyone starts looking out for themselves so it’s not them that gets the blame and productivity will suffer.

I think it is more important to put in place processes to minimise issues happening again. If a deployment causes downtime, don’t ask who’s fault is it and sack them. Instead what can we do to reduce the chance of it happening again?

As someone who has an interest in DevOps, I often break things and don’t want to get blamed for that, I do want to improve my processes so I am not always breaking the same things.

What do you think about Blame? Is what can we do differently always better than who did this?

Interfaces

Mon, 27 Mar 2017 20:00:45 +0000

I am trying to understand interfaces and when to use them in my code.

An interface defines a contract and any class that implements that interface agrees to fulfil that contract.

Lets look at an example as this tends to be how I learn best.

Most applications require some sort of data to work from so lets start by defining IData which can load data.

public interface IData
{ 
  Blog LoadData(string Connectionstring);
}

My interface defines one method LoadData and outputs an object called Blog (I will explain why in a minute)

A common data source is a SQL database so we could define a SQL class that implements IData.

public class SQL : IData
{ 
  public Blog LoadData(string Connectionstring) 
  { 
    Blog blog = new Blog(); 
    using (SqlConnection con = new SqlConnection(Connectionstring)) 
    { 
      con.Open(); 
      //etc

We could also get data from an RSS feed of a blog (hence why I called the object Blog earlier)

public class XML : IData
{ 
  public Blog LoadData(string Connectionstring) 
  { 
    XmlDocument myXmlDocument = new XmlDocument(); 
    myXmlDocument.Load(Connectionstring); 
    Blog blog = new Blog(); 
    foreach (XmlNode RootNode in myXmlDocument.ChildNodes) 
    { 
      //etc

Both classes implement IData and have a method called LoadData which has a string parameter and outputs a blog object. The string parameter is either a connection string to a SQL database or the URL of the rss feed. Not sure if there is a better way of doing this bit, maybe the name of the string needs making more generic.

Now we have some classes that implement an interface what can we do with them. Lets write a class called GetData which gets data but doesn’t care if it comes form the rss feed or a SQL database.

public class GetData    
{        
  private IData _repo;         
  public GetData(IData repo) 
  { 
    _repo = repo; 
  }         
  
  public Blog LoadData(string Connectionstring)        
  {            
    var original = _repo.LoadData(Connectionstring);            
    return original;        
  }    
}

When we call GetData we can either pass in XML or SQL as the class is not tied to either implementation. We could even write other classes that implement IData for testing purposes.

My full code can be found on github .

The advantages of writing code in this way include code that is easier to extend, easier to test and easier to maintain. This is only the start of my understanding and I am sure this is going to be a topic I come back to in the next few weeks.

SOLID and other programming terms

Mon, 20 Mar 2017 20:00:45 +0000

This week I have been looking at improving my understanding of a few programming terms, like SOLID and I will try and define them so I can look back here when I get confused.

MVC

I have previously blogged about MVC , but my understanding was not 100% correct so I will refine this here.

Model – Now this is where my understanding was not quite correct. I thought the model was the actual source data, eg an XML file, SQL database etc. The model is the business logic so this is a processed version of the source data. MVC does not care where data is stored it can be flat files, SQL, XML or anything really.

View – This displays data to the user and typically is the HTML/CSS markup. Only display related logic would get included in the view.

Controller – This is another place I had got a bit confused. I had thought all the logic lived here. This is incorrect controllers are only concerned with getting data between Model and View.

SOLID

SOLID are five principles of object oriented programming and design.

S is for Single Responsibility Principle. A class or piece of code should be responsible for doing one thing.

O is for Open/Closed Principle. Code should be open for extensions but closed for modifications. Often this refers to the way you can implement an interface and add extra functionality.

L is for Liskov Substitution Principle. Objects in a program can be replaced with subtypes of that object with out changing functionality.

I is for Interface Segregation Principle. Large interfaces should be split down into small interfaces so that clients only know about methods that are of interest.

D is for Dependency Inversion Principle. High and Low level modules should depend on abstractions.

I need to look into SOLID some more but here are some examples .

DRY

DRY is simply Don’t Repeat Yourself. It is often easy to spot when a function call will help you not have to use the same bit of code in different places. I am often spotting places where I can put this into action.

Test Driven Development or TDD

Mon, 13 Mar 2017 20:00:45 +0000

A few weeks back I attended a talk at Agile Yorkshire about Test Driven Development or TDD by Dr Oliver Shaw. I was impressed at how easy Oliver made it look, so as I have never tried it I thought I should give it a try.

Test Driven Development or TDD is a way of development which starts with writing a Unit Test. First you write a failing test, then you write the code to make it pass, then you refactor your code. This can be remembered by thinking of Red, Green, Refactor. Red being the failing test, Green being getting the test to pass, and Refactor being the refactoring.

During the demonstration Oliver used a language called scala and a system that automatically reran all the tests after every change. I code with Visual Studio in C# is there a way I can get my tests to run automatically as well?

A bit of googling and configuring I can answer this as Yes.

The nuget package called Giles is a watcher which will rerun your tests similar to how Oliver did it with his scalar environment. Fans of Buffy the Vampire Slayer will get the joke of why a watcher is called Giles. I couldn’t get this to work with MSTest but works fine with NUnit. There is a powershell script giles.ps1 which you need to run and will update every so often with how many tests have passed or failed. However you may not see this if you are coding in Visual Studio but there is a way to get a notification.

If you install the application Growl you can get notifications from Giles which pop up and then disappear. So whatever you have on screen you can find out almost instantly if you have broken tests.

Another thing that I wanted to configure is a way of viewing code coverage and which methods are tested and which aren’t. If you are familiar with VSTS after a build it gives you a percentage score for test coverage. I don’t find this overly useful as it doesn’t tell you what is covered and what isn’t. Also what if you want to use Github, how do you calculate the code coverage then?

The nuget packages OpenCover and ReportGenerator allow a html report of code coverage to be produced. I created a batch script that can be run whenever you require this information.

[path]\OpenCover.Console.exe -target:"[path]\nunit3-console.exe" -targetargs:"[path]\Test.dll" -output:"[path]\coverage.xml" -register:user

[path]\ReportGenerator.exe "-reports:[path]\coverage.xml" "-targetdir:[path]"

The commands are fairly straightforward, the only tricky bit is sorting out all the filepaths to the different programs.

Now that I have all this plumbing setup time to give TDD a try and see what I can build.

Github Vs Bitbucket Vs Visual Studio Team Services

Mon, 06 Mar 2017 20:00:45 +0000

As a developer using source control and git is bread and butter of what we do. Github is probably the most popular and widely known hosting service for source control but I have also used Bitbucket and Visual Studio Team Services. Lets have a look at each one and what they offer. Note while I have included prices I have only tried out the free versions.

Github

URL: https://github.com/
Private Repositories: Not Available for free
Public Repositories: Unlimited
Team Size: Unlimited
Prices: $7 per month for unlimited private repositories, $25 per month for 5 users then $9 per month per user

This is probably the most widely used service for hosting code. Over 13 million repositories of code. This is an ideal solution if you want your code to be publicly viewable, but take care not to publish passwords, private keys or your companies trade secrets. Every developer should have a Github account for displaying bits of code they are proud of.

There are a number of externally built APIs that link into Github for doing extra features, like building, code coverage etc

Bitbucket

URL: https://bitbucket.org/
Private Repositories: Unlimited
Public Repositories: Unlimited
Team Size: Less than 5
Prices: $10 per month for 10 Users, $100 per month for 100 Users, $200 per month for Unlimited Users

At my last job we used Bitbucket extensively for all our projects. All the code was private so only the team could access it, however before I left we were approaching the 5 user limit (but looking at these prices cost seems very reasonable)

Visual Studio Team Services

URL: https://www.visualstudio.com/team-services/
Private Repositories: Unlimited
Public Repositories: Not Available
Team Size: Less than 5
Prices: $30 per month for 10 users, and other features paid for via Azure Invoices

Visual Studio Team Services or VSTS is Microsoft’s version control solution and I have only just started using it, however what I have seen I like. There are lots of options for building your code so VSTS is more than just hosting your code it is verging on a Continuous Integration/Delivery solution. Being a Microsoft product there are numerous links to Azure and it is easy to deploy stuff to that platform.

All three have options for tracking issues but VSTS have options to add Stakeholder users which would allow none developers to add and keep track of issues and progress with them.

If I want to run tests, look at code coverage VSTS is probably the solution I would go for, if I want something that is public I would go for Github. What do you think which of these is your favourite?

Learning R

Mon, 27 Feb 2017 20:00:45 +0000

Today I spent some time learning the R language.

The problem I was trying to solve was to convert local prices of some items into Euros. I had been using a fixed exchange rate for all data, but as exchange rates fluctuate so much this is incorrect.

My first though was to find a free API that I could query to get the values I wanted. The first API I found didn’t cover all the currencies, the next one I found I burnt through the free allowance in one pass.

A colleague of mine mentioned using R to solve this, he sent me some links and I set out to write my first piece of R code.

My finished code can be found on github and I will attempt to explain some of it.

R defines functions fairly simply

nameoffunction <- function(arg1, arg2)  
{  
  arg1 * arg2  
}

I have created a function that takes 2 parameters date and currency. I know I have about 10 different currencies that I want to get currencies for and I want to loop through each day so I will need to pass in a date.

The source of my exchange rate information is the www.xe.com website, its historical exchange rate page passes currency and date into the query string so I should be able to build up a string containing all the different elements.

All programming language can concatenate strings and R is no different. R uses paste()

var <- paste("Hello", "World")

However R has an annoying feature in this function. I would expect that var in the above example would contain “HelloWorld”, it doesn’t it contains “Hello World”. Why it automatically adds a space I don’t know?

var <- paste("Hello", "World", sep="")

I am not entirely sure what all of the code does but I can take a good guess.

read_html() I would guess loads a html page, html_nodes() finds all the html tags of a certain type on the page, in my case <table>, html_table() reads the first table it finds.

table1[2] selects the second column, and head() selects a specific number of rows. I want the first row and second column so I combine these two as head(table1[2],1)

Now that I have found my exchange rate what do I do with it? R can read and write to SQL Server so why not store this info in a SQL lookup table. I can then use this data in a stored procedure when I process my data.

To query sql you can use sqlQuery(), it has two parameters, a sql connection and a TSQL command (eg a SELECT, INSERT, UPDATE statement)

I use a while loop to loop through every day between 1st October 2016 and today and look up the exchange rate for each currency.

For now I am manually running this R script, however there are ways to run R directly from SQL Server which I may well investigate. I could then have a SQL job to run this on a schedule, maybe once a day to get the latest exchange rates. I also would like to do something a bit cleverer like only getting exchange rates for the days that I need them by querying existing database tables.

Side Project – Connect 4

Mon, 06 Feb 2017 20:00:45 +0000

Hello?

You have probably like me almost forgotten about this blog. Life has got in the way over the last few months, but lets see if I can restart my blogging habit.

In 2017 I want to start a side project for a few reasons. I want to improve my coding skills and look at things I wouldn’t normally during my day job.

I have been trying to decide what to build. The actual project doesn’t matter too much, it is the techniques I use during build that matters the most. I would like something that I can start off as a windows app and then extend onto the web or mobile apps.

I started off thinking about a Rubik’s cube app, however my initial coding has lead me to believe it is probably too complex for my first app.

Last year I spent an evening looking at Connect 4 during a York Code Dojo session. I think this should be complex enough that I can code some logic to efficiently solve the game, however not too complex I get defeated before building too much.

So far I have built a console app that displays via Console.Write() commands, a grid to represent the game, with 0 meaning empty, 1 red and 2 yellow etc. I am currently working on the logic to solve the game, once I have that in a good state I can extend into a winforms app, into a web (MVC most likely) app and finally a mobile app (I have no knowledge about how to do this yet, but something I would like to try one day).

Over the last few days I listened to .NetRocks where they discussed Progressive Web Apps . This sounds like a great challenge for me to aim for. A progressive Web App is a website that automatically detects various properties to give you an app that runs well whatever the state of your internet connection and whatever browser you are in.

I have lots to learn but I can split it down into small sections and I can build something that demonstrates some of my skills.

What should be in Source Control?

Thu, 24 Nov 2016 20:00:45 +0000

I am currently working on source code that is over 5Gb in size. This is mostly due to a poorly thought out folder structure, there are code files, images and Excel files all jumbled together. I think a clear distinction should be made between source code and data.

Source Code

I will define source code as anything that is written in order to compile and run the project. If it is a webpage it will be all the HTML, CSS and Javascript or any file used to produce these. I would also include any configuration files and files used to build/deploy the website or project. Anything that is compiled from your source files can safely be ignored.

Data

I would define data as anything that is added to the project during its life. So if you have an upload option, anything that is uploaded I would describe as data. The site should still function without (or very little) data.

Images

Images can fit into both groups. Any icons or images attached to the functionality of the project I would class as source code. However anything that is uploaded should be classed as Data.

Databases

The database should also be classed as both. The data, anything that is inside a database table should normally be classed as data. Stored Procedures, Functions and Views are all Source Code and would benefit from version control.

Source Control != Backup

Source control is not an excuse not to backup things. Don’t just commit files to source control so you know you can restore them if you need to. Files in general in source control are there so you can see how they changed over time as the code base changed. Files in you backup are a snapshot of what the application was at a point in time and will include ALL the data.

One last point before I end. If you are hosting on a Cloud Computing platform like Azure it gives you an easy way to distinguish between Data and Code.

Anything in your

Web App = Code
Blob Storage = Data
SQL = Data/Code

Each project is unique and there will always be exceptions to these suggestions but I think this is a good goal to have. What do you think?

Getting control of a codebase

Thu, 17 Nov 2016 20:00:45 +0000

So recently I started working on a new codebase. I will be honest when I first saw it, it was a mess. Here are a few of the things I did to try and regain control.

I was given access to the source code on Visual Studio Team Services. However this consisted of a single commit 3 months ago. When I looked at what was running on the production server it was clear that changes were being made live with no regard for source control.

The first thing I did was commit everything that was running live into source control.

Next I created a SQL Server Data Tools (SSDT) project to keep track of all the database objects. Previously there was a folder with some stored procedures in it, but these did not match with what was currently running.

I now had in source control the current state of the website and the database, so I knew I could get things back to this state if I made some bad changes.

Lets start by looking at the website code I had. There was no solution file, the only way to look at the website was to setup my local IIS to run what was in the website folder. I could then use Visual Studio to “open” my local IIS website and attach to process to debug it.

Next I Looked at Default.aspx to see how the website worked. The majority of the website code was stored in the database stored procedures. After the tag there was a <% %> which contained a Response.Write(RunSP.RunStoredProcedure(Parameter1, Parameter2, …) command, which executed a stored procedure and the results of the stored procedure was the html code including any javascript that the webpage needed to display. I will be honest I have never seen any code like it. My guess is that the developer was secretly a DBA and wanted to make any web page changes by just changing how the stored procedures work.

This meant that the website is not going to do anything without a backup of the database running, and meant my SSDT project was going to be vital. However the database was in a bad state, it consisted of a fair few broken objects and SSDT would not build.

Using find I went through each of the broken database objects to find where in the code they were being used. Luckily most were referenced in commented out code, so I just removed all the broken database objects. The database could now be built. However there was a dependency on the users table of another database. (This was the developers solution to sharing logins between websites) As I was using SSDT I added a database dependency, problem solved for now.

Next I tried publishing my database. SQL CMD encountered a parsing error. The reason for this was my SPs contained javascript eg $(document), SQL CMD uses $(DatabaseName) as variables for different database so it was getting itself confused.

My solution was to use Find and Replace to replace all the $ with ’ + CHAR(36) + '

So I now have a SSDT project that builds and publishes but still no website project.

To get the website running from Visual Studio I started off creating a .Net 4 website project and added Entity Framework 5 and MVC 3 via nuget. I then copied all the website code into the new project, and with a bit of work I got it to build. Most of the work was relating to namespaces and referencing the correct one and moving the EF model from AppCode to a custom named folder. A bit of trial and error later I had a version of the website that could be run from Visual Studio.

I have not deployed my new version of the website as it needs further testing. No automated testing or even a smoke test checklist currently exist.

As my source code is hosted on Visual Studio Team Services (VSTS), I can get VSTS to build each commit and check I haven’t broken the build. This is not that helpful at the moment, hopefully one day I will have automated tests that can be run here as well.

Wow, I feel like I have done loads with this code so far but there is loads more still to do. I need to understand more about the business processes behind the code with a hope to understand why some architectural decisions have been made. I want to refactor the code as much as is possible, I would like to remove much of the html/javascript from the stored procedures as I can’t see that there is any advantage to running a website like this. Please correct my if I am wrong.

LINQ

Thu, 06 Oct 2016 20:00:45 +0000

What is LINQ?

LINQ is an acronym for Language Integrated Query, which describes where it’s used and what it does. The Language Integrated part means that LINQ is part of programming language syntax. In particular, both C# and VB are languages that ship with .NET and have LINQ capabilities.

How do I use LINQ in my C# code?

To use LINQ the first thing you need to do is add the LINQ using statement.

using System.Linq;

In your code you need a datasource, for this example I am going to use a simple array, but it can be anything eg SQL, XML etc

int[] data = new int[10] { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };

Next you need a LINQ query. (Note I know the Q in LINQ means query, so I have just written query query, if you are one of those people who hates seeing PIN number you might not like this blog post.) A LINQ query is very similar to a T-SQL query, so if like me you are good with databases this should make sense to you.

In T-SQL you can have:

SELECT num
FROM data
WHERE num = 1

In LINQ this becomes:

var query =
from num in data
where num == 1
select num;

Finally you need to do something with the query you have written. I am just going to print the results of my query to console.

foreach (var num in query)
{
  Console.Write(num);
}

What other SQL like syntax can I use?

In T-SQL you can control ordering using ORDER BY, LINQ has a similar syntax orderby

orderby num descending

In T-SQL you can use GROUP BY, to do something similar with LINQ

group num by num.Type into type
select type

foreach (var type in query)
{
  Console.Write(type.Key);
  foreach (var num in type)
  {
    Console.Write(num);
  }
}

JOINS

So you thought joining tables was a SQL Server only thing. Think again you can do this in LINQ

var joinquery =
from cust in customers
join prod in products on prod.CustomerId equals cust.Id
select new { ProductName = prod.Name, CustomerName = cust.CompanyName };

Conclusion

There are loads more LINQ functionality that you can use. While writing this blog I found https://code.msdn.microsoft.com/101-LINQ-Samples-3fb9811b which has loads of examples of different queries that you can write with LINQ.

This has inspired me to use LINQ more in my code and learn more about the different queries that could be written.

Looking back Ten years

Thu, 29 Sep 2016 20:00:45 +0000

Next month I will celebrate ten years working at my current job, two weeks after that I will start a new chapter of my life at a new company. Lets take this opportunity to look back ten years at some of the great stuff I have achieved.

2006

In 2006 I had no IT experience. If you were to ask me where Event Logs could be found I wouldn’t know, and the letters DNS meant nothing to me. I was quiet and shy but hard working. The idea of making IT my career hadn’t occurred to me, this was just a job.

I started off life doing first line support and compiling health and safety files onto CD-ROMs (known by the company as Eurofiles) with HTML. At the time the company had a mix of Windows 2000 and Windows XP, using MS Office 2003. Server wise I can’t remember exactly but I think two domain controllers, a database server and a backup server using a tape drive. At this point in time I don’t believe I did a lot with the servers but I think the servers were running Windows Server 2005.

As time went on I learnt more and more about what the company did and how stuff worked.

Nagios

One of my early achievements was setting up a nagios server. I have blogged about Nagios before, but it is a server monitoring system that runs on linux. I am extremely proud of what I have achieved here, no one else in the company had my knowledge of how nagios worked. Initially I even experimented with using a cheap mobile phone connected to the serial port (remember when PCs had these?) to send SMS messages to alert of down time. However this was abandoned when I burned through all the credit on this phone.

Nagios is still used today, and my current team have been shown how to extend the system as the business changes. My philosophy has always been if we have a problem that Nagios didn’t warn us about we need an extra nagios test. These days the nagios web interface is publicly available on the internet so can be checked from anywhere (assuming the office internet is up!) and a mobile app replaces the SMS idea we originally had.

2011

At the end of 2010 the IT Manager resigned, then the Developer resigned and finally the Lead Developer resigned. I was alone. I was IT Manager. I negotiated my first pay rise since joining the company. I was IT.

Wow things were stressful back then. I had so much to learn but somehow I managed. I learnt how to Interview, yes I have employed some clangers along the way, but also some great staff. As well as internal staff I learnt to deal with contractors. We had contractors to help with our internal systems and also contractors to do development work. One thing I have learnt about contractors is that you will always have to chase them at some point to deliver what you are paying them for.

Server Migration

Around the start of 2011 our email server started to show its age. Exchange 2003 had a hard limit on the size of its information store and we were rapidly approaching this. Along with an IT contractor I worked to migrate to Exchange 2010, this was a huge project and caused all sorts of issues which we just worked through. Since then I have done other migrations so Windows Server 2012 and also virtualized many servers in 2013.

Bandwidth

A major limit with our infrastructure has always been the internet connection coming in. For a while we tried to load balance three ADSL connections, but the upload speed was always a limiting factor. It was a major victory for the simplification of our network when we got a leased line installed into our head office, helped by a government grant we gave us free installation. Not content with this I did the same again for our second York office, including the free installation.

Learning Development

Since I started I have always been learning development stuff. But in the past few years I have learnt a massive amount helped largely by two main things. My boss passing lots of my responsibility onto others and dedicating lots of my time to development tasks, and also the opportunity to learn with our outsourced development team.

Finishing off

There are a few things that have been ongoing that we have wanted to change since I started which I can’t really take the credit for as they are not complete yet, but I am proud as they may be finished in the next few months.

The company stores calendar information in one giant excel spreadsheet, this is being replaced by outlook calendars.

Our last Windows Server 2003 server and tape drive is being decommissioned. This is the only server that has been running for the entire time I have been here.

The way the company keeps track of work coming in and who does what is being reviewed. Over the years many people have tried to adapt the old database that one of the directors created in Access many years ago. I did a major overhaul recently to delete unused columns and added extra invoicing functionality. What is needed is a fresh system, maybe a CRM can do everything they want. I wish them lots of luck in doing this as I know it won’t be easy but it has to be done.

2016

There are many other projects and pieces of work that I have done that I am proud of. In 2016 I will leave a stable IT department. All clients are on Windows 10 and using Office 2010. We have virtual servers running Server 2012 and are using some great services from Azure to run our SaaS websites. There are of course things I would have liked to achieve, and things I wish I had done differently but on the whole it feels like the right time to move on, especially now long running projects are starting to conclude.

10 Ways to Survive as an IT Manager

Thu, 22 Sep 2016 20:00:45 +0000

So after five and a bit years of being an IT Manager here is some advice I have learned along the way in no particular order. On the whole I have enjoyed myself but it has been a real challenge at times.

1. Figure out what plates are still spinning

Being an IT Manager is all about keeping everything running all of the time. A bit like spinning 5 or 6 plates. You have plates for your servers and network infrastructure, you have plates for bespoke databases that you maintain, you have plates for your staff (including any external contractors), you have plates for any websites or apps that you develop. That is a lot of plates to keep spinning and that before you start thinking about what your boss wants you to deliver. Make sure you know what is happening with all these plates, which ones are happy, which ones are on the way to the floor and which ones you need to get the glue out and repair.

2. Make it someone else’s problem

If you can blame someone else do so. If your internet goes down it is your ISPs fault. If your website dies its your hosting company’s fault. Take responsibility for problems but if when something goes wrong you can pick up the phone and ask for help, it will make your life easier.

3. Hire good staff

Hiring poor staff wastes time and money and makes you look bad by others. Demand the highest salary band for new staff that you can afford and don’t agree to hiring anyone that you have doubts about. It is easy to bow to the pressure to get someone quickly but this will always result in worse problems in the long run. Once you have a good team do your best to keep them, and warn upper management of the problems if staff leaves (basically make it their problem not yours!).

4. Learn, Learn, Learn

You may or may not have the opportunity to go on training courses. Whatever your situation spend time learning new stuff that will benefit the company and yourself. You can learn a lot by reading online, you can petition for training from your managers, you can fund training yourself, you can ask for help from your different suppliers. The more you learn, the more you can do and the more useful you can be to the company, plus the more interesting you will find the job.

5. Say No!

Don’t be afraid to say no. You will always be asked to do the impossible and if something is impossible say so at the start. It wastes everyone’s time if you spend a lot of time trying to do the impossible. Always give your reasons for saying no, and if you always say no people will think you are unhelpful. A better way to say no is to come up with a better solution. No I can’t do it your way but here is a better solution.

6. Don’t give estimates

If you are asked how long something will take you don’t answer straight away or give an exaggerated estimate. Go away and spend some time thinking of everything that is involved before replying. There will always be something that you forgot to consider when first asked about it and looking at the different components will help plan out the work needed as well as provide an estimate.

7. Know what to tell your boss, and what not to

This is a hard one to get the balance right for. You need to tell your boss enough so that they appreciate all that you do, but too much and they will stop listening and accuse you of talking in technobabble. I have never got the balance right with this one. I have always aired on the side of not telling my boss enough, and hence they don’t realize that I saved the day on Sunday night as everything is working again on Monday. Do repeat yourself. If your server is running low on resources start asking for replacement hardware early, and increase the frequency and the panic in line with the problems it is causing.

8. Understand the problems of the business

Businesses need to make money. If the one you work for isn’t making enough money you will soon be looking for another. If you work for IT you will quickly start to see the problems of the business, think about what simple changes IT could make to improve things that would benefit the whole company. Some of your suggestions won’t go anywhere, but some may have a massive impact. I can think of a few changes that IT have spearheaded that I am very proud of, upgrading our internet connection, simplifying or automating processes and delivering new versions of software.

9. Ask for help

Don’t be afraid of asking for help. There are lots of places to look for help. Other departments could take more on, you could recruit extra help, you could hire external contractors. You can ask questions on support forums like ServerFault or StackOverflow , many software re-sellers or other suppliers are a good point of contact for questions about things they supply. Microsoft Support was also invaluable for a server issue.

10. Think about disasters

Write a disaster recovery plan or backup policy. Yes there will always be something more important that needs doing, but just stop for a moment to think how you would feel if everything died on your watch. The one thing you can rely on with technology is that it will fail at some point. A back of the envelope plan of action is better than no plan at all, even better is a detailed plan of what to do when each and every service you rely on fails. Plan additional services with an idea of adding extra redundancy. Always have multiple Domain Controllers, think about what data you could run from the Cloud. VMs could be replicated to the Cloud, and servers could be run from there.

Imposter Syndrome

Thu, 15 Sep 2016 20:00:45 +0000

This week I handed in my notice at a job I have had for almost ten years. In a few weeks time I will start a brand new job as a web applications developer.

This is a great achievement for me and a great chance to learn and expand my development skills.

However moments after I handed my notice in, a huge wave of uncertainty enveloped me. I have read about this and I think I was suffering from an attack of Imposter Syndrome.

Despite external evidence of their competence, those with the syndrome remain convinced that they are frauds and do not deserve the success they have achieved. Proof of success is dismissed as luck, timing, or as a result of deceiving others into thinking they are more intelligent and competent than they believe themselves to be.

I started to convince myself I couldn’t do the job I had just accepted and that moments after I arrived on my first day I would be found out as a fraud and kicked out the door.

My logical brain was no match for the Imposter Syndrome. Having been kicked out the door of this new job, I would become penniless in no time with no employer (even my previous boss wouldn’t want me back) or recruitment agency willing to speak with me.

OK lets see if we can fight off the Imposter Syndrome and look at a possible worse case scenario. I start at my new job, it is very hard work and a month or so later I have to move on.

No employer ever would sack me on my first day. My first day will be lots of learning how they do things, looking at procedures. I have lots of experience of learning new things, looking at procedures and if I put the work in I can make this new job a success. I have lots of good ideas and useful experience that no one at my new company has.

I don’t know what the solution is to Imposter Syndrome other than to try and ignore it by thinking positively and the knowledge that I have done amazing things in the last ten years and I will do the same in the next ten.

Happy 50th Star Trek

Thu, 08 Sep 2016 20:00:45 +0000

On September 8th 1966 a TV show called Star Trek first aired. Today 50 years later people around the world are still watching Star Trek.

I was going to blog about the Star Trek story, but instead why not watch this video.

DZone

Thu, 25 Aug 2016 20:00:45 +0000

For a while now I have been sharing some of my blog posts on the Dzone website.

The Dzone website allows users to submit links to content and I have been submitting the content I have created on this website. This is how Dzone describes themselves:

With over 1 Million members, DZone.com is one of the web’s largest communities and publishers of technical content for software professionals. Developers from all over the world come to DZone for the latest and best content to hone their skills and advance their careers.

Well this week I have been invited to join the MVB (Most Valuable Blogger) programme. The hope is that more of my readers will find my content from the DZone website.

The DZone team will soon start sharing my content on the DZone website and I hope this will result in even more people reading what I have to say.

If nothing else this is clearly telling me that my blog is making a difference, people are taking note of me and what I have to say. I need to keep going and keep writing articles and be more consistent.

Amazon Web Services Pt 2

Thu, 04 Aug 2016 20:00:45 +0000

Last time I started looking at Amazon Web Services and how it differed from Azure. I am going to continue looking at what it can do.

Virtual Machines

Lets look at what you can do with Virtual Machines. I selected to create a new Virtual Machine (or as AWS calls them an EC2 Instance)

First you choose a name for your VM and then the OS that runs on it. There are 5 main OSes to choose from Windows Server, Amazon Linux and a selection of the most common Linux flavours.

You can then download a certificate to secure your VM.

Like Azure, AWS takes a few moments to create your VM. While I wait I can see that AWS has configured a firewall so only my current IP can connect to it.

Once the VM is ready you can download an RDP file. However to get the login details you need to decrypt the password using the certificate you downloaded when you created the VM.

It is interesting to compare the difference in security between Azure and AWS. Azure allows the resetting of passwords of VMs directly from its console, however I suspect that in AWS if you loose your certificate (AWS states they don’t keep a copy of this) you would have to recreate your VM.

Like with the Websites the default name of the VM is much less user-friendly than what you get from Azure. However I suspect there are other options I haven’t spotted that may customise these.

Azure Portal vs AWS Console

I really like the Azure Portal. It feels like something that has been designed so you can easily access all the options for a specific Azure feature.

The AWS Console probably has all the same options as with Azure however I don’t think it looks half as good, and will take me a while looking through menus to find the equivalent options. Part of this is due to my unfamiliarity with AWS, so will get easier with time.

Amazon Web Services

Thu, 21 Jul 2016 20:00:45 +0000

I am a big fan of Azure but I know zero about its biggest rival – Amazon Web Services or AWS.

So lets sign up for a free trial and see what it can do.

The AWS free trial is available from https://aws.amazon.com/free/ and lasts for 12 months. From memory I think the Azure free trial lasted only one month.

To start you need to login with your amazon account and create an AWS account. This requires your name and address and your payment info (you will only get billed if use services not covered by your free trial).

Interestingly AWS requires you to verify your identity via an automated phone call. (I don’t recall doing anything like this for Azure but please correct me if I am wrong.)

Once you are logged in you get a series of links displaying all the different services that are available. First impression is this is a simpler view to Azure’s portal with a similar amount of services. At the top right is an option to select which region you want to use, in Azure I use North Europe and West Europe, AWS has Ireland and Frankfurt.

Create a Web App

First thing to try is setting up a website. I selected create a web app and I get a page asking me for its basic details (very similar to Azure, however AWS asked what language your code is written in, Azure handles all of these) AWS websites appear to support a host of different options similar to Azure.

The actual creation of your website takes a few moments (like on Azure). However the default URL for websites is similar to http://test.vjbbimyv7w.eu-central-1.elasticbeanstalk.com/ which is not quite as nice as the Azure equivalent http://test.azurewebsites.net

Azure has a host of command lines available via powershell. AWS has a similar command line interface called AWS CLI, including the option to deploy from git to your website.

AWS Toolkit for Visual Studio is an extension that allows for the publishing of websites to AWS. (Just like you can publish to Azure)

As I learn more about AWS I will continue to blog about it. Amazon Web Services Pt 2

Clever things with MS Access

Thu, 07 Jul 2016 20:00:45 +0000

I hate MS Access and especially developing with it as you can’t do any thing clever with it.

Wrong, Wrong, Wrong!

There are a few clever things I have been able to script to make developing with it passable. I still would rather use Visual Studio but this improves the experience a fair bit.

Source Control

Access files are binary (I use the ADP/ADE file format but I assume other Access file formats have the same problem) and so you can’t diff them to see what has changed. This is bad.

However there is a solution to this. A tool called Access SVN and can be downloaded from https://accesssvn.codeplex.com/ , this gives you a way to extract to text files all the forms and reports that are in Access. Before every commit I would manually run this tool on my ADP file and extract to text files, then I would commit these text files to source control and could easily see what had changed in each commit.

Despite the name Access SVN, the tool is not tied to subversion, you can use any source control system, I use git.

Also included in this tool is a way to do this with the command line, so you can make this a build step on your build server. I have not used this extensively yet, but the syntax is fairly simple

asvn.exe e "path to Access file" "path to txt files" "*.*"

The filter at the end . allows you to specify what to extract so you could extract all forms/reports beginning with D with “.D”. I had trouble using . because the names of my forms/reports contain characters not allowed in a windows file name. I am sure there is a way round this but I haven’t had chance to look into it further yet.

Testing MS Access

Surely testing is not possible with MS Access? I would have agreed with that statement the other day until I found a neat way of testing if a feature is enabled.

Firstly a bit of background. I develop using MS Access 2003 because the design view is far easier to use, however because it is out of support all my users use MS Access 2010. MS Access 2010 has a feature called Tabbed Documents which allows all forms and reports to open in new tabs so you can easily switch between them. This feature can only be enabled in MS Access 2010 and has no effect if opening with MS Access 2003.

If you use Access SVN on your Access file with tabbed documents turned on and off you will see UseMDIMode: 0 and UseMDIMode: 1 show up in the Database properties file. UseMDIMode: 0 means that tabbed documents is turned on.

In powershell I can now write a test to see if UseMDIMode: 0 can be found in the database properties file

Get-Content "General\Database properties.dbp.txt" | Select-String "UseMDIMode: 0" -quiet

If the test passes True will be returned, if it fails null will be returned.

On my build server I scripted the extraction of Database properties.dbp.txt from the ADP file with asvn.exe before running this test. While not strictly needed as Database properties.dbp.txt should be in source control, it is possible that someone could forget to extract the text files from the ADP, with this step you are always testing what is enabled in the binary file.

MS Access Connection Strings

While developing with MS Access I often swap the database connection to point to my local machine or a build server. I always try and remember to only ever commit with this set to the live database to avoid obvious problems.

The other day I found on stackoverflow a way to script this. I love this! I can include this step in my deployment process and it will overwrite what ever the connection string is in source control with what your production environment needs.

All you need to run this step is, (note it is spaces between the parameters not commas)

cscript connect.vbs Project.adp "ServerName" "DatabaseName"

The contents of connect.vbs can be found on the stackoverflow article . It is also possible to pass username and password if your environment requires this.

Compiled ADE

The last clever thing I do with MS Access is convert my ADP file into the compiled ADE version. To manually do this there is an option in the tools menu.

To automate this I run

cscript createADE.vbs "path to ADP" "path to ADE"

The contents of createADE came from this forum post , the only change I made was to comment out some of the echo statements so it would run silently as part of my build process. It should be noted that cscript and wscript are almost identical and either will run these scripts however in a command line environment cscript is preferable, and wscript should be used in a windows environment.

I am quite surprised at how much I have managed to do in terms of scripting the build and deployment process for MS Access. I still don’t like developing with Access but this has definitely improved things.

Periodic Table of DevOps

Thu, 30 Jun 2016 20:00:45 +0000

The periodic table lists all the chemical elements and groups them together based on some key properties. Today I found an article about the periodic table of DevOps .

I am not going to discuss every element but I thought I might go through some that I have heard of or used.

Github – The repository of lots of open source software. My Github can be found at https://github.com/funkysi1701
Amazon Web Services – The second most popular cloud computing provider (not sure why this isn’t number 3?)
Git – The distributed source control system that everyone uses these days.
Azure – The number one cloud provider, I have used this a lot, mostly with websites but also with some of their other features like Traffic Manager.
Bitbucket – like Github but allows private repositories. I have used this extensively for work based projects that I don’t want to be public.
Google Cloud Platform – don’t know much about this one, but no surprise that google wants a piece of the cloud computing pie.
Selenium – This is a product I want to play about with as allows front end testing with a browser.
Rackspace – Before we made the jump to Azure we made use of some Rackspace servers.
Subversion – The first source control system that I used, but been using git so long now not sure I can remember how it worked.
Visual Studio – The IDE from Microsoft that I use to write code. I am a big fan as it does everything I could want.
TeamCity – The continuous integration software that I have been using to automate my deployments.
MSBuild – This is used by Visual Studio to build your software and can also be used by your deployment scripts.
Trello – A website that allows you to create a board of ideas or things to do.
Slack – Brings all your communication together in one place. It’s real-time messaging, archiving and search for modern teams.
New Relic – A software analytics tool suite used by developers, ops, and software companies to understand how your applications are performing. Useful but find myself favouring Application Insights (part of Azure) more now.
Nagios – Yay nagios is on the list! My favourite server monitoring system.
Splunk – This application can be used to search, monitor and analyse all your log files to find out what is happening. Don’t currently use it but I have tried it out in the past.

What is your favourite DevOps tool? Why not leave a comment below?

Automatic Git Tagging

Thu, 16 Jun 2016 20:00:45 +0000

One of the features of git is the ability to tag a point in my change history with a tag. For a while now I have been manually tagging my code whenever I do a release, so I can easily work out what has changed by doing a diff between two tags.

Now that I am automating my release process with TeamCity I am thinking about how to manage my tags better.

TeamCity has a setting called VCS Labeling which comes in very handy.

Configuring it is fairly simple as it only has three settings.

VCS root to label: This is obviously the url to your git repository Labeling pattern: This is the text of the label to be added. Label successful builds only: Do you really want to add a tag if the build failed?

A tag needs to have a unique name, so adding a tag just called deployed won’t work. When I used to add tags manually I used the naming convention of deployedyyyymmdd. While this naming convention is possible with TeamCity I use something a bit more complex to provide more information about what has been deployed.

TeamCity provides lots of parameters that can be used in your build steps and also in the Labeling pattern box. I started off using deployed-build-%system.build.number% as my tag which just marks git with the TeamCity build number.

When I run a TeamCity deployment I don’t always use the same configuration options, I deploy locally, to a test server or to production and sometimes I just deploy the frontend or the backend. How cool would it be to include this information in the tag text?

Well the next step was to change my Labeling pattern to deployed-build-%system.build.number%-%ServerName%-%DatabaseName%-%FrontEndPath%, this adds the backend database config settings and the path the frontend was deployed to. Now when looking at git you can see commits marked with multiple tags, one for each deployment that succeeded and the tag will indicate the settings used during that deployment.

Now I will never forget to add the tag after a release as the adding of a tag is part of the deployment process, if the deployment fails the tag won’t be added. I can test my deployment to test and git will show if this has been successful, and when I deploy live this will also show up.

How do you use tags? Do you mark successful builds with a tag? Why not let me know or leave a comment below.

I’m 100 blog posts old

Thu, 02 Jun 2016 20:00:45 +0000

That’s right this is the one hundredth post that I have written on this blog.

So what have I learned in the past 100 posts?

It is easier to write a blog before you become a parent. More recently I am increasingly finding it difficult to find the time to blog. It used to be that I could write on an evening, but now James is around I often prefer to play with him, or more often stop him crawling where he shouldn’t.
I like my job. The inspiration for most of this site is my day job and as you can read, I do a wide variety of different things, but I have a lot more to learn as well.
Finding your niche is hard. 100 posts in and I am still not sure what my niche is. I started out with the broad niche of IT and what I do, I then considered something about IT and fatherhood but I don’t think that topic is really me. My current thinking is maybe DevOps especially as my role these days fits squarely between Development and Operations.
Its time for a refresh. I have been meaning to change the theme of this site for some time and I feel after 100 posts now is as good as time as any. I want to emphasise my skills and what I am learning and increase the emphasis on DevOps, which I think will be my niche.
Watching visitor numbers is addictive. Every day I look at how many people have looked at my blog, but I have yet to see a pattern between what I write and how many reads I get. Is my writing getting better? I don’t know. Are more people reading? Probably not. Will I keep going? Yes

So what is next?

Hopefully a refreshed look in the next few months. Hopefully regular posts. If there is something you want to see on here drop me a message via any of the social media links or put a comment below.

What is my favourite post?

Maybe User Groups and F# which proved very popular and got me to start going to user groups something I still enjoy today. I also like Coding Myself Into A Corner which got me to start thinking more if I was giving myself future problems. But there are many others I like such as James and Becoming a father

Brothers, TNG S4 E3 Review, The Battle Bridge

Thu, 26 May 2016 20:00:45 +0000

My latest podcast can be found http://www.trekmate.org.uk/brothers-tng-s4-e3-review-the-battle-bridge/ and feedback can be left on the trekmate forums http://forum.trekmatefamily.com/2016/05/brothers-tng-s4-e3-review-the-battle-bridge/

Data jeopardizes an emergency mission to save an ill child when he gets a signal from his creator.

Today James and Lou discuss (Brothers) and what we think of it, with tonight’s guest host Simon Foster, Funky Si.

Credit: Main Title Theme (TV Edit) was arranged by Dennis McCarthy and composed by Jerry Goldsmith and Alexander Courage

Running Windows on Raspberry Pi

Thu, 05 May 2016 20:00:45 +0000

Last year you may remember me talking about playing with a Raspberry Pi. Well since then my Raspberry Pi has been sat on a desk collecting dust.

This week I attended Leeds Sharp and the topic was Running Windows on Raspberry Pi and this has inspired me again to do something with a Pi.

But first what did I learn.

Here are a couple more of pics of last nights @LeedsSharp https://t.co/QvlsYjNFvB #RaspberryPi #MSIoT pic.twitter.com/60o4wiiPSv
— Richard Tasker 🇬🇧 (@ritasker) April 29, 2016

I had heard that a cut down version of Windows 10 could be installed on the newer Raspberry Pi’s, but I hadn’t really understood how cut down the version of windows is. Having now seen it demonstrated the OS consists of a single page with a few menu options.

The real power of Windows 10 IoT is when you connect remotely to it. There are a couple of ways to do this, PowerShell (check out https://ms-iot.github.io/content/en-US/win10/tools/CommandLineUtils.htm for a few commands), and of course connecting Visual Studio to your Pi.

When I had previously played with a Pi, it had been with bash scripts and linux commands. The beauty of installing Windows IoT is that you can write c# code, something I do in my day job so theoretically I should find it easier.

The demonstration at Leeds Sharp was pretty impressive. If you are a fan of the Big Bang Theory you may recall Sheldon playing a Theremin. Well it is actually possible to construct a Theremin from a couple of sensors and a Raspberry Pi. The code for which is on github .

Now that I have been inspired what shall I do?

My Raspberry Pi won’t support Windows 10 IoT, so I need to buy the latest version. I am thinking of buying a kit so I can play about with a breadboard, LEDs and resistors. Maybe not build a robot straight away but certainly try doing something that connects to the GPIO pins.

If you have any suggestions leave a comment below.

Exchange Web Services

Thu, 21 Apr 2016 20:00:45 +0000

Where I work we use a really old fashioned way of recording where in the country employees are: Excel!

For years I have been trying to persuade staff to use calendars in Exchange. Outlook is great for looking at one or two people’s calendars at once but quickly gets unmanageable for looking at ten or more peoples availability.

Recently I have started looking into how easy it is to query this information to give a custom view.

Microsoft provide an API to query exchange information called Exchange Web Services or EWS. I have only used EWS with my Exchange 2010 setup, but the documentation mentions working with Exchange 2007 and older or Exchange online.

Here are the basics of what I have tried. Fire up Visual Studio and from nuget install EWS.

Install-Package Microsoft.Exchange.WebServices

I started off with a simple Console App to see how it all worked, and then extended it to a MVC website. I found querying exchange directly was slow, but it is easy enough to cache information in a database.

To start you need to create an Exchange Service object, by specifying the version of Exchange you are using.

ExchangeService service = new ExchangeService(ExchangeVersion.Exchange2010);

Next you need to pass the URL you are using to access Exchange.

service.Url = new Uri("mail.example.com");

To access information from exchange you need to pass some Exchange credentials, ideally a username and password that has access to view all the calendars you want to look at.

service.Credentials = new WebCredentials("username","password");

Next pass the email address of the user who owns the calendar you want to look at.

service.ImpersonatedUserId = new ImpersonatedUserId(ConnectingIdType.SmtpAddress, "me@example.com");

My particular exchange server has a self signed SSL certificate which is not going to be trusted by remote clients. The following line ignores this validation check and makes my program connect.

System.Net.ServicePointManager.ServerCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => true;

Now that we have connected to exchange we just need a few lines to look for events in the calendars.

// Initialize the calendar folder object with only the folder ID.
CalendarFolder calendar = CalendarFolder.Bind(service, WellKnownFolderName.Calendar, new PropertySet());

// Set the start and end time and number of appointments to retrieve.
CalendarView cView = new CalendarView(startDate, endDate);

// Limit the properties returned to the appointment's subject, start time, and end time.
cView.PropertySet = new PropertySet(AppointmentSchema.Subject, AppointmentSchema.Start, AppointmentSchema.End);

// Retrieve a collection of appointments by using the calendar view.
FindItemsResults<Appointment> appointments = calendar.FindAppointments(cView);

Now that you have an appointments object you can loop through each element with a foreach loop. In my case I assign each elements Subject to a variable, which I can then do what I like with (display in a console window, save to a database, display in an MVC website.)

foreach (Appointment a in appointments)
{
    if (a.Subject != null)
    {
        subject += a.Subject.ToString();
    }
}

My website queries a SQL database which I can easily populate with a console app that runs at regular intervals throughout the day.

There is a lot more I want to do with this project as this is only the basics of what you can do with Exchange Web Services. So expect more blog posts on this subject as I expand its functionality and learn new ways of doing things.

Revisiting Team City

Thu, 24 Mar 2016 20:00:45 +0000

Last year I blogged about Team City , well I have been looking at it again recently. In that time they have even changed their logo!

Lets start with thinking about what I want my Continuous Integration server to do.

Check out my code from source control (usually master but all feature branches would be even better)
Configure specific setting for build
Build my code
Build my databases
Run any unit tests
(Optional) Run deployment to Azure Test/Live site

There are probably other things I want to achieve but I will start with these six.

Checking out code from source control is something Team City does out of the box, so I can safely say I have done this now. It even monitors a branch for changes and initiates a new check out.
Team City allows you to create specific build steps so in theory you can have multiple builds for every variation of settings that you want for your code. I have not tried this yet apart from building with the default config, but I don’t expect it will be too difficult.
I have managed to get my code to build with Team City, it took a bit of tweaking the different build steps but wasn’t too difficult. Team City has a visual studio build agent which takes you solution file and does what it needs to. The one problem I have found with this step is that I get errors with my tests if I select a Debug config instead of Release.
Databases are always the problem part of the deployment. So far I have manually deployed my databases but I intend on revisit this step. A stackoverflow post suggests that I can run SQL code via Team City in the following way by creating a command line executable:

Command executable: c:\Program Files\Microsoft SQL Server\100\Tools\Binn\sqlcmd.exe
Command parameters: -S [ServerName] -i [PathToSQLScript]

I have yet to try this but I am hopefully that it will just work. Dropping a database and restoring a back and then running different SQL scripts is all possible from TSQL, so I should be OK. Watch this space for more details.

Running the unit tests got me stuck for a while. I tried setting it up using VSTest or MSTest neither worked mainly because a config file wasn’t being copied with the test binaries. When I tried using NUnit it just worked. The tests that failed gave me a few config settings to change.
I have powershell scripts that deploy to Azure websites, I think that these could form the basis of a deployment to Azure. Again the difficult step here may end up being deploying all the different databases to Azure. This is also the riskiest step as I need to connect to live servers which is why I will leave this to last, at the very least I could generate scripts that do a full deployment.

That’s it for now. Once I have this all working I will revisit again with details of the database steps as I am expecting a few challenges to overcome. What have you used a CI Server for? Are there other things I want to achieve from a project like this? Why not contact me or leave a comment below

Model View Controller (MVC)

Thu, 17 Mar 2016 20:00:45 +0000

Model View Controller or MVC is a software architectural pattern for implementing user interfaces on computers. It divides a given software application into three interconnected parts, so as to separate internal representations of information from the ways that information is presented to or accepted from the user.

I have been trying to get my head around the concept of MVC for a while, hopefully writing this article will help solidify my understanding of it.

One of the core concepts of MVC is the ability to separate concerns so you can concentrate your energies on one aspect of the application.

Model This is the data. If your application uses a database the model often mirrors what you have in the database and concerns itself retrieving information from the database.

View This concerns itself with displaying the data to the user. Typically this is the html pages of your application.

Controller This concerns itself with actually doing things and deals with user interaction. Typically it will get data from the view and send data to the model.

The three concerns can be developed in isolation as they do not depend on each other, for larger development teams you can even divide up development much easier that a traditional app.

The basic concept of MVC I get and understand, however I find myself getting bogged down in the details.

The database doesn’t matter. I need to remember this and not get sidetracked in writing custom methods to connect to the database which end up unmanageable. I know SQL, so can easily write SQL commands to copy data into the format I need for my app. The app I am currently working on involves a large amount of existing data, and I need to concentrate on the MVC parts and worry about the database later.

In previous attempts I have tried to build my model against many tables, but instead I can write a query against many tables and insert that into one table which the Model can then use.

Changing the model often results in an error informing you that the context has changed since the database was created. The easy solution to this in my case is to drop the database and allow entity framework to recreate the database each time. As long as my database contains no new data, I won’t loose anything.

One of the core advantages of MVC is the ability to test it or even use test driven development (TDD). I haven’t really dabbled with testing yet as I am still trying to get my head around the fundamentals, but once I have made some progress with my app I want to test, so next time I get asked to add a new feature I have no fear about breaking stuff.

For the first time I have got an app with a working Authentication system from the start. And it is remarkably easy to implement with one keyword. Adding [Authorize] to the top of your controller is all that is needed. Building the actual Authentication system is relatively easy from Visual Studio, as it has templates for Azure AD, Forms Based, Open Auth like google/twitter etc.

It is still very early days for my MVC app and my understanding of it, but I feel I have turned the corner and can actually build something with it now, rather than be stuck in a downward spiral of confusion.

What do you think about MVC why not leave a comment below? For more info about MVC I have been looking at http://www.asp.net/mvc which has more information and tutorials.

Surface Book Review

Thu, 10 Mar 2016 20:00:45 +0000

I have just bought myself a new laptop, but it is not just any laptop it is a Microsoft Surface Book .

I think this is probably the first time I have bought myself a top of the range laptop and after a few days of use I am loving it.

The Surface Book is the latest in Microsoft’s Surface line of tablets but the first to feature a keyboard and be more like a traditional laptop. The keyboard is detachable from the keyboard so you can use the Surface Book like a tablet.

As it is a top of the range laptop, it was not cheap, so if price is a significant factor in your laptop choice this is not the machine for you. Microsoft are trying to compete with Apple’s Macbook range.

So what did I get for my money:

Windows 10 Pro
16Gb RAM
512Gb SSD (Formatted Size nearer 474Gb)
6th Generation Intel Core i7 Processor
Surface Pen
13.5-inch touch screen

I have only had the device for a few days but these are my thoughts so far. My primary reason for buying this is for doing development work with Visual Studio and SQL Server.

I really like the Surface Pen. This is a feature that I didn’t think I would use much. The Pen connects via bluetooth and allows interaction with the touch screen. The pressure sensitive screen allows all sort of touch actions to be performed.

I especially like the choice of keyboards that are offered when the physical keyboard is detached. You can have either a on screen keyboard which you can type with pen or fingers, an on screen keyboard that splits in half or you can write with the pen. This is an amazing feature. My handwriting is not good but most of the time it understands my scrawl. With this option you can hand write a tweet or fill in a form on a webpage. For speed I would not recommend this form of input for large amounts of text, but for browsing the web of when you are just clicking on links it is great.

Windows Hello – I can unlock windows just by looking at my screen. How cool is that! It was really easy to setup, it just takes a photo of your face and next time you login all you need to do it look at the screen. Note if you are working in tablet mode, make sure the tablet is the correct way up.

It’s not all brilliant though. Detaching the screen is fiddly and sometimes takes a few moments to do. A few times I have felt I needed another hand but I am sure the more I do this the easier I will find it to do.

Battery life isn’t great especially when running off the tablet only. This is due to the machine having two batteries, one in the base and one in the screen so with keyboard attached you have much longer use times. Actually I am finding the battery is lasting longer now that I have used it for a few days.

The screen has a very high resolution 3000 x 2000 but using clever zooming technology everything is still readable and not tiny. However I use Remote Desktop a lot and this caused me a problem. When RDPing the remote session used the host screen resolution which made everything tiny on my servers. The solution to this can be found on SuperUser and involved using Microsoft’s Remote Desktop Connection Manager, a cool bit of software for managing multiple RDP sessions. This is actually an improvement on the way I usually work, but until I found the answer this was annoying.

Another minor annoyance with the keyboard is that you can’t press Ctrl-Alt-Del with one hand. Normally this isn’t a requirement but if the screen locks while I have my son on my lap I am stuck, but I won’t blame the surface book for this.

When I got my Surface Book I was also given a Microsoft Wireless Display Adapter. I am not a fan of this bit of tech as I can’t get it to work. I briefly got my old laptop to connect but my Surface Book keeps telling me NO!

Overall I like the Surface Book. It is certainly the nicest laptop I have ever used. Now that I have the docking station I can connect two monitors, have a wired internet connection and it becomes a proper work horse. What does everyone else think? Better than Apple’s range of laptops?

Trying Out Azure Active Directory

Thu, 03 Mar 2016 20:00:45 +0000

One of my plans is to create new MVC Webapps for my companies databases. Once I publish these I will need to secure them so only staff have access.

The traditional way to do this would be insert membership tables into my database. The user then has to remember another username and password and I have to secure the storage of these credentials. Lots of work for everyone.

There is a better way by using Azure Active Directory. You have probably heard of Active Directory, if you are a SysAdmin you probably use it all the time to manage your corporate users and computers. Azure Active Directory is an extension of this into the Cloud.

I have blogged in the past about using Azure but this is the first time I have tried connecting my internal domain to Azure. There is a Virtual Lab which helped me try out some of these ideas.

The first thing I did was to create a new Directory in Azure. I did this via the old portal (manage.windowsazure.com ) it might be possible via the new portal but I don’t know how yet.

Click New, select App Services > Active Directory > Directory and select Custom Create. Select Create new directory, give it a name and a domain name and select a region from the drop down. Then add a Global Admin for this directory.

There is a tool called Azure Active Directory Connect . Download and Install this with express settings on one of your domain controllers. You need to specify a domain admin account to access your domain and the Azure Global Admin account you just created.

At this point I went to bed so I am not sure how long it tool to sync all the domain information but by morning it was all showing in the users list on Azure.

All my user accounts are showing with a @contoso.onmicrosoft.com, it is possible to use custom domains but I haven’t figured out that step yet. I made a change in my Active Directory and a while later that change was showing in Azure AD.

So now what? Open up Visual Studio and see if I can use Azure to Authenticate.

I selected to create a new MVC web project and clicked the change authenticate option. One of the options was Work and School Accounts, I then selected Cloud Single Organization and entered contoso.onmicrosoft.com. I then ran this app and it authenticated using Azure using my domain password. Really impressed at how easy that was.

The app then shows up on Azure in the old portal. In Applications you can see a list of which users have access to your app and configure few other app related settings.

This is a long way off being useful in my actual app, but it shows that the basics of what I am trying to do does work. Anyone done anything similar with Azure AD? How did you get on?

Coding myself into a corner

Thu, 25 Feb 2016 20:00:45 +0000

I spend an awful lot of my working life adding new features and improvements to a legacy database system.

A legacy system is an old method, technology, computer system, or application program, “of, relating to, or being a previous or outdated computer system.” Often a pejorative term, referencing a system as “legacy” often implies that the system is out of date or in need of replacement.

My particular legacy system is Microsoft Access. We use Microsoft Access to provide the front end for all our internal databases. The particular technology we use is ADP files which are only supported in Access 2000, 2003 or 2010.

Microsoft has already dropped support for Access 2000 and 2003, and support for 2010 is due to be dropped in 2020.

So why am I coding myself into a corner? Its simple for every feature or improvement I create in Access, I increase the amount of features I need to create or port to a new front end.

This is a very depressing thought, everything I create will need creating again. I am not saving myself work but increasing the amount of work I need to do again.

The bad news is that my employers like almost all businesses demand results and like a good employee I have been delivering them. I have been promised the mythical “when we are quiet” you can work on rebuilding the database front end. I know this will likely never happen so what are my options.

Do nothing I have warned my managers that this work needs doing and it is up to them to give me the resources I need. This is not an acceptable option. Firstly I am not future proofing the business, 2020 will be here before we know it. Also I am not developing myself as a developer, as the only experience I am getting is with Legacy technology that has expired or will do very soon. No one wants to employ someone who only has legacy experience.

Do something I need to keep delivering results and doing what is asked of me. Like any good engineer I should be multiplying my estimates by four, meaning that I have some time that can be used for looking at the bigger picture. This is a win-win option really. Employer gets a solution that is future proof, Employee gets valuable experience in up to date technology.

What do you think? Have you been tied to legacy technology? Why not leave a comment below.

Star Trek Episode Review The Best of Both Worlds

Thu, 18 Feb 2016 20:00:45 +0000

Episode Title: The Best of Both Worlds
Star Trek Type: TNG
Original Air Date: 18 June 1990 & 24 September 1990
Written By: Michael Piller
Directed By: Cliff Bole

The Borg threaten the Federation, when they kidnap Captain Picard and destroy a fleet of 39 Starships. Luckily some original thinking from Commander Riker and Lieutenant Commander Shelby saves the Federation from assimilation.

Thoughts: I love this episode! It is probably my favourite episode in the whole of trek. I can’t think of a single scene which I don’t like.

It is a two parter that actually works best as a two parter. The moment that Riker orders the Enterprise to fire on the Borg is great. At the time some thought that Patrick Stewart might leave the show and be replaced by Riker/Shelby. I am very glad that didn’t happen but it is interesting to ponder what that might be like.

The Borg soundtrack in this episode is great. Ron Jones really captures the spooky threat that the Borg are. I was always disappointed that some of that didn’t continue in First Contact and when the Borg return in Voyager.

The episode has a cracking pace. From the moment the Borg first attack the Enterprise to the moment the Borg ship explodes the show doesn’t slow down.

I am Locutus… of Borg. Resistance… is futile. 
Your life, as it has been… is over. From this time forward… you will service… us.

Commander Shelby is a great character that enhances the episode and introduces conflict between characters, particularly Riker.

I have heard that Riker’s thoughts about his career was a mirror to writer Michael Piller’s thoughts about leaving the show. Luckily for us he stayed.

In a future episode of DS9 Worf comments that he felt there was nothing that they couldn’t do during this time. I have to agree the tag team of him and Data is a great one. If I want rescuing it would be Data and Worf that I would want.

My good friends at Trekmate have just reviewed part one, why not have a listen?

My git repository is too large!

Thu, 11 Feb 2016 20:00:45 +0000

Today I did a clone of one of my git repositories and it took ages to download. Looking into what got downloaded it was easy to see why. The .git folder was over 500Mb in size.

I know how this has happened. This repository was created in 2013 and has been used by me as a dumping ground for lots of things related to the code which never should have been committed.

Since 2013 I have learnt a lot more about coding and git so the current version of the files in git isn’t too bad. But git keeps the history of changes for every file so bad practices like this are kept.

What can I do about this? Well what does google suggest? I found this blog post .

It suggests ways of listing all the large files that are stored in git and a way to remove them. As I am the only person that regularly commits to this repository I see no problem with giving it a go.

I will summarise the steps here.

git clone url

Now you need all the remote branches so there is a bash script to run

#!/bin/bash
for branch in `git branch -a | grep remotes | grep -v HEAD | grep -v master`; do
git branch –track ${branch##*/} $branch
done

Another bash script then lists the top 10 large files

#!/bin/bash
#set -x
# Shows you the largest objects in your repo’s pack file.
# Written for osx.
#
# @see http://stubbisms.wordpress.com/2009/07/10/git-script-to-show-largest-pack-objects-and-trim-your-waist-line/
# @author Antony Stubbs
# set the internal field spereator to line break, so that we can iterate easily over the verify-pack output
IFS=$’\n’;
# list all objects including their size, sort by size, take top 10
objects=`git verify-pack -v .git/objects/pack/pack-*.idx | grep -v chain | sort -k3nr | head`
echo “All sizes are in kB. The pack column is the size of the object, compressed, inside the pack file.”
output=“size,pack,SHA,location”
for y in $objects do
# extract the size in bytes
size=$((`echo $y | cut -f 5 -d ‘ ‘`/1024))
# extract the compressed size in bytes
compressedSize=$((`echo $y | cut -f 6 -d ‘ ‘`/1024))
# extract the SHA
sha=`echo $y | cut -f 1 -d ‘ ‘`
# find the objects location in the repository tree
other=`git rev-list –all –objects | grep $sha`
#lineBreak=`echo -e “\n”`
output=“${output}\n${size},${compressedSize},${other}”
done
echo -e $output

After running the script you will see details of your largest files. I had *.msi and *.exe files in the mix. To remove them run the following, where filename is the path to the file that needs removing.

git filter-branch –tag-name-filter cat –index-filter ‘git rm -r –cached –ignore-unmatch filename’ –prune-empty -f — –all

To reclaim the disk space run the following commands

rm -rf .git/refs/original/
git reflog expire –expire=now –all
git gc –prune=now
git gc –aggressive –prune=now

Now push your changes back to the remote server.

git push origin –force –all
git push origin –force –tags

Now if you do a clone it will be much smaller than before and you can get back to coding much quicker, without having to wait.

SQL Transaction Log Backups

Thu, 04 Feb 2016 20:00:45 +0000

Like many DBAs I spend a lot of time maintaining my SQL Server backups.

From SQL Server I maintain both full backups and transaction log backups. I have often restored my full backups but until recently I have never restored a transaction log backup. All backup strategy’s are only as good as the last time you tested the restore process.

So what is a transaction log backup?

A transaction log backup contains all the transaction log records generated since the last full backup and is used to allow the database to be recovered to a specific point in time (usually the time right before a disaster strikes). Since these are incremental, if you want to restore the database to a particular point in time, you need to have all the transaction log records necessary to replay database changes up to that point in time.

How to do the restore.

First right click on Databases in SQL Management Studio and select restore database. You should then get a screen similar to this.

In source click the … to allow you to select your backup files.

Now normally I have only ever selected one file here, the *.bak file. Instead select the *.bak and all the *.trn files as well. After SQL Server has chugged for a few minutes (time will depend on number of transaction files and server/disk speed etc) the restore plan section should fill up with files.

In the destination database box, type in the name of the database you want to restore. I recommend using a different name to avoid overwriting the original database, appending Test or a datetime to the name is what I usually do.

On my test server I need to untick the take tail-log backups option off the options screen before I can execute the restore.

Now you can either check the tick boxes in the restore plan section or (more fun) click the timeline button to select at what point in time you want to restore to.

You can either select the point in time with your mouse or specify the exact point in the time textbox. Alternatively you can just select the most recent point, probably the most likely option when disaster strikes.

Now that I have tried doing this on my test server I feel much more confident that when disaster does strike I can get things restored quickly and painlessly.

How often should you run transaction backups?

The answer to this question depends on how critical your data is. Until very recently I ran mine ever 15 minutes, I have increased this to every 5 minutes, but I have seen recommendations of running it every minute . The more critical your data the more often you should run them.

Development Annoyance

Fri, 29 Jan 2016 20:00:45 +0000

I spend a lot of my time creating new features that simplify my companies business processes.

A good example of this is an invoicing system I created. Instead of users working off different spreadsheets and copying and pasting data between various different programs the user can click a couple of buttons and everything is done and they can move on to the next task.

I am currently expanding this system so more of the companies work can be invoiced quickly and simply.

I like working on this kind of problem.

I get to discuss with the different departments involved, finding out how they work and what could make their lives better.

I then get to look at what structures already exist in the database and get to rip out anything that makes no sense (Today I had a Customer table which has a primary key called ClientId – its now a Client table with PK ClientId) and add new structures to store new information.

I then can build a user interface so the users can interact with the data.

But the most important part of the process comes next. This is when I show the users what I have built and how it will make their lives better, they can then feed back to me questions and comments. How do I do x?, what about y?, have you thought about z?

After this very valuable feedback I can go back and tweak what I have done making it better and better until the users think it is suitable for what they need.

Now why did I call this post development annoyance?

I find it intensely annoying that after spending weeks of my time working on creating something to be used. Users either make do with my solution and never tell me the one thing that really annoys them or worse they go back to the time consuming process and never feedback the one or two changes I need to make to complete the project.

Now I am not perfect, like many IT people my people skills are somewhat lacking but apart from asking people what can you do to get the feedback you need to complete the project?

I do not have the knowledge that other departments have so I consider development a team effort that requires everyone to contribute otherwise it will not finish.

The second outcome where users return to their old ways is the most annoying. It could well be months or even years before I find out that what I created is not being used any more. By that time my memory of what I did and more importantly why I did what I did has been lost and forgotten.

So users please, please tell me what works and what doesn’t and what things I can do to improve things. I want my solutions to constantly improve and get better and I can only do that with your help.

Star Trek Episode Review Arena

Thu, 21 Jan 2016 20:00:45 +0000

Episode Title: Arena
Star Trek Type: TOS
Original Air Date: 19 January 1967
Teleplay: Gene L Coon
Story By: Frederic Brown
Directed By: Joseph Pevney

The episode begins with the Enterprise in orbit of Cestus III. Commodore Travers requests a tactical team join the away team. When the away team beam down Cestus III has been destroyed.

Spock detects non-human life signs, obviously the aliens that destroyed the outpost. They attack the Enterprise, stranding the away team who are under heavy fire. Sulu is instructed to do whatever he can to protect the Enterprise and leaves orbit.

Kirk finds the armoury and a grenade launcher, after firing the enemy withdraws.

After beaming back aboard the Enterprise they pursue the unknown enemy into uncharted space. One of the survivors of Cestus III explains that the enemy attacked without warning.

Kirk is determined to destroy the enemy ship before it reaches home and the Enterprise prepares for battle.

An unknown solar system scans the Enterprise as they continue their pursuit of the enemy ship. The enemy ship comes to a complete stop. The Enterprise prepares to attack but before they can they also come to a complete stop. The Metrons have stopped both ships because both ships were on a mission of violence. The captains of both ships will be placed on a planet where the winner can destroy the other.

Captain Kirk and the enemy captain appear on a desert planet. The enemy is a reptilian like species called the Gorn. The Gorn has superior strength to humans but Kirk manages to escape his first encounter.

"This is Captain James Kirk of the Starship Enterprise. 
Whoever finds this, please get it to Starfleet Command. 
I’m engaged in personal combat with a creature apparently 
called a Gorn. He’s immensely strong. Already, 
he has withstood attacks from me that would have 
killed a Human being. Fortunately, though strong, 
he is not agile. The agility and I hope the cleverness, is mine."

The Metrons said that the planet contained materials to allow the building of a weapon, however at first glance Kirk sees nothing that could help him defeat the much strong Gorn captain. He does find some diamonds, too small to be used as a weapon.

Kirk pushes a huge boulder over a cliff onto the Gorn captain, which appears to knock him out, however he survives and Kirk barely escapes. Kirk then finds Sulphur, which triggers a memory.

The Metrons allow the Enterprise to watch the last stages of Kirks struggle. They see Kirk discover Potassium Nitrate. The Gorn accuses the Federation of trespassing into their space.

Kirk starts constructing a weapon using the diamonds, sulphur and potassium nitrate to make a gunpowder explosion to fire the diamonds at the Gorn captain. Just in time he applies a spark to his weapon and his opponent lies defeated.

Kirk refused to kill the Gorn captain as they may have been just defending themselves. The Metron announces that Kirk has displayed the advanced trait of mercy and there may be hope for humanity after all.

Thoughts: I like this episode very much. It has some classic Star Trek ideas in it.

We have a super powerful alien race (the Metrons) that think there is hope for humanity.

We have Kirk using raw materials to construct a weapon using simple chemistry.

We have an enemy race that appears destructive but may have just been defending themselves.

We have the classic location of Vasquez Rocks, somewhere that if I ever visit the US I have to visit.

We have the very cheesy Gorn mask, yes its not realistic but the story is strong enough that it doesn’t really matter. Enterprise brought back the Gorn but as a CGI character and I think I may prefer the rubber mask.

Interestingly the weapon Kirk builds would likely not work. The US show MythBusters proved that the bamboo wasn’t strong enough to hold the explosion and the weapon would likely wound Kirk as much as it would wound the Gorn.

Score: 10/10 Redshirt Count: Cestus III Outpost, 1 Red Shirt and 1 yellow shirt from away team.

Disaster Planning

Thu, 14 Jan 2016 20:00:45 +0000

Over the Christmas break the city of York experienced the worst flooding disaster it has seen in recent years. If you want to help donations are being taken at York Flood Appeal .

Luckily I wasn’t affected but it was really sad seeing many of my favourite streets full of water. Thousands of people had to be evacuated from their homes when the water was at its highest.

York has its phone exchange located in the centre of the city and as the flood waters rose, this building flooded. BT who run the exchange worked flat out trying to restore phone and broadband services to York and the surrounding area, but this lead to at least 24 hours or more of downtime for users.

During this time most of the city wasn’t able to take card payments from customers (due to an internet connection being required), cash machines were not working for the same reasons and the most worrying thing access to emergency services was not available.

Anyone would think the city phones were down #yorkfloods #bthardatwork pic.twitter.com/ZavLMsG2Js
— Simon Foster (@funkysi1701) December 28, 2015

I am in no way critical of the hard working engineers that worked on the BT exchange to restore services to the city. However I have to ask what was contained in the BT disaster recovery plan.

As an IT Manager myself backups and disaster planning is something I need to think about and plan for. Recent investment in our internet connection meant that we are now on a leased line so BTs downtime would not have affected us in terms of network connectivity, I am uncertain about phone but I believe as a business we could have continued running during this time if we needed to.

Planning for a disaster is not something I ever make enough time, however I definitely want to spend more time on it and I do have the basics of a plan in my head which is probably more than BT has.

BT is a huge company they run almost the entire country’s phone lines and according to wikipedia have operations in 170 countries. I think that they can afford to have someone plan for disaster recovery.

The York Exchange is located next to a river that often floods. What could the company have done to avoid this problem?

Relocate some or all of its services to a location away from the river.
Add the ability to reroute services through an alternative exchange.
Many towns and villages surrounding York use the BT exchange surely some of these could have been configured to use other exchanges.

In these days of terror alerts and bomb scares. Imagine if York was London and all it took to take down communication ability of the capital was one Exchange to be offline?

I am sure BT are looking at what they did right and what they did wrong and if you are reading this have a look at your own disaster recovery policy. Don’t get caught out like BT did.

Top 50 Star Trek episodes

Thu, 07 Jan 2016 20:00:45 +0000

In 1966 a TV show called Star Trek first aired in the USA. Since then 726 episodes have been made and 12 films have been released. 2016 marks the 50th anniversary and Star Trek is as alive as ever with a new film due out this summer and a brand new TV series in the works for broadcast in 2017.

But what can I do to celebrate? With that many episodes I don’t have enough time to re-watch every episode so how about a re-watch of the top 50 episodes.

Star Trek has five live action TV series so all I need to do is pick my top 10 from each.

Star Trek

The original Star Trek series first aired on September 8th 1966. It lasted only three years and 79 episodes. This series featured the iconic characters Kirk, Spock and Bones on a five-year mission to explore space.

The City on the Edge of Forever
The Devil in the Dark
Balance of Terror
The Trouble with Tribbles
Arena
This Side of Paradise
Amok Time
Space Seed
Mirror Mirror
The Deadly Years

Star Trek: The Next Generation

The first spin-off series set 100 years after Kirk featured a new Enterprise continuing to explore space. This series aired between 1987 and 1994 for 178 episodes.

Best of Both Worlds
Yesterdays Enterprise
The Offspring
The Measure of a Man
Disaster
All Good Things…
Tapestry
The Inner Light
Cause and Effect
Unification

Star Trek: Deep Space Nine

The second spin-off series which ran at the same time as TNG was set on a space station so instead of exploring adventures arrived via a stable wormhole. This series aired between 1993 and 1999 for 176 episodes.

Emissary
Duet
The Way of the Warrior
The Visitor
Sacrifice of Angels
Waltz
Trials and Tribble-ations
In the Pale Moonlight
Far Beyond the Stars
Siege of AR 558

Star Trek: Voyager

The third spin-off series ran after TNG and featured a female captain and a starship stranded in a remote part of the galaxy journeying home. This series aired between 1995 and 2001 for 172 episodes.

Scorpions
Year of Hell
Caretaker
Message in a Bottle
Eye of the Needle
Heroes and Demons
Jetrel
Timeless
Pathfinder
Riddles

Star Trek: Enterprise

The most recent spin-off ran after Voyager and was set 100 years before Kirk. This series aired between 2001 and 2005 for 98 episodes.

Broken Bow
The Andorian Incident
Shuttlepod One
In a Mirror Darkly
Twilight
Damage
Azati Prime
Regeneration
Impulse
Zero Hour

I am not going to review each episode here, I will save that for future blogs. With 50 episodes that is one a week to get them all reviewed during Star Trek’s birthday year. We will see if I manage to keep to that schedule.

A look back at 2015

Thu, 31 Dec 2015 20:00:45 +0000

Wow! What an amazing year 2015 has been and with 2015 about to end I thought it would be good to look back at what happened in this amazing year.

January

I started the year with a declaration that 2015 would be my year of code. Well I have had a few breaks from programming but I have learnt loads this year. I have flitted a bit concentrating on different areas of programming but I think that has broadened my knowledge. I consider myself to be a programmer now , which I didn’t at the start of 2015.

February

A first for 2015 was appearing on a podcast . The Trekmate podcast Ten Forward were the lucky ones that got to experience my first attempt at talking to the world. This continued throughout the year with appearances on Upper Pylon 2 , The Battle Bridge and even the Sci-Fi Waffle podcast.

It was a sad time for Trekkies at the end of February when Leonard Nimoy passed away.

March

In March I talked about my recent efforts with Azure Traffic Manager to help prevent downtime. I also spent a lot of time improving databases which led me to consider ways to deploy it better.

Also in March I attended an IT networking event called ITBoss . Free food and drinks and a discussion with fellow IT Managers.

April

April was a bit of a mixed bag. I celebrated my second wedding anniversary but I also said goodbye to my car due to financial reasons.

I played about with a Raspberry Pi , mostly with the camera . I really must make some time to play with it more.

May

In May I went to a C# User Group called Leeds Sharp, was great fun and I really need to make more time and go back in 2016. It did result in one of my most popular blog posts .

Usually developers don’t enjoy finding bugs in their software but I had great fun when I realised I had a Stack Overflow in one of my databases, it was easy to fix once I worked out what was going on.

In May the UK went to the polls as we had another General Election.

June

In June I celebrated my 50th Blog Post . Amazing to think I had already written 50 Blog posts, even more now!

Leeds Sharp had a coding challenge, to solve Sodoku puzzle programatically. Didn’t get it finished but made a good stab at it.

July

In July it was SysAdmin Day and I celebrated with my staff the amazing work we do keeping everything working.

Microsoft launched Windows 10 in July, the latest operating system is being offered for free for a year.

August

In August everything changed as my son James was born. I took time out from blogging and work during this time to spend with James.

Before James was born I stepped down from my involvement with the technical team at St Michael le Belfrey.

September

In september it was back to work, where I started planning how to add internet connections to reduce the chance of outages.

October

In October I took James to a dads group . This was the first time Dad spent time with James without Mummy and it was awesome.

I also celebrated my first birthday after becoming a father, was great to have a card from James. James also got the chance to spend time with my family in Nottingham.

November

In November the countdown to Christmas began with James appearing in his first Nativity as Jesus and dressed up as an Elf at a Christingle service.

Exciting Star Trek news was announced this month, a new TV series is to be aired in January 2017. No other details yet but very exciting.

December

December is all about Christmas and we had a great Christmas time as a family.

Sadly York was hit by Floods just after Christmas, luckily my family was not affected but many others have been. Hopefully by the time you read this the cleanup will be well underway.

Before Christmas I spent my time counting down the days with the Advent of Code

What will 2016 bring?

I don’t know. There will be lots more time with James as he continues to learn and develop. I will continue to learn and develop my coding skills. Watch this space as I keep blogging about it.

Christmas 2015 with the Fosters

Thu, 24 Dec 2015 20:00:45 +0000

Four months ago James David Martin Foster was born. And he is even more awesome now than when he was born.

As I write this James only has a few sleeps (as James sleeps multiple times in a day I can’t put an exact number) until his first Christmas. James is pretty much unaware of what is going on as he is still too young but that doesn’t stop his parents enjoying the season.

He has lots of Christmas themed clothes which we have been enjoying taking pictures of him wearing. He makes a great looking elf I think.

We took James to his first Christingle service at St Michael le Belfrey at the start of December. A Christingle service is where you stick a candle and cocktail sticks into an orange. James loved it, he already loves looking at lights so loved looking into the light from the candle.

Another Christmas activity we did this year as a family was to appear in a nativity. St Michael le Belfrey every Christmas hold a cafe to tell people the good news about Christmas and they have a nativity outside which includes donkeys. Don’t we make a great holy family. (Excuse the coffee cups but it was cold.)

Me and James’s mummy have gone a bit crazy with buying him presents, we have both got him lots of things to open on Christmas morning (or more likely mummy/daddy will do the opening)

James also has three Christmas days to look forward to, with different sides of the family each no doubt with him as the centre of attention.

Exciting times. Happy Christmas everyone, make yours as special as ours.

The Christmas count down with code

Thu, 17 Dec 2015 20:00:45 +0000

In December we count down the days to Christmas. People often have advent calendars filled with chocolates or even Lego.

But what about if you are a programmer, how can you count down the days till Christmas? What about if you had a daily Christmas themed coding challenge? Well www.adventofcode.com is the answer.

Each day a new challenge is available. So far the challenges are related to Christmas and Santa delivering his presents. The challenge is split into two closely linked parts. You are given a randomly generated input and your need to give an answer using a short string or number.

The challenges are getting harder, with some completely defeating me, (not surprising with my limited programming knowledge). The challenges are language independent so you could try them in javascript or c#. I have been using Visual Studio and C# and my code is viewable on github .

So far I have completed Challenges 1 to 6 and also 10. Partly because I found challenge 7 very difficult but I am hoping to have another go at it.

While coding through these challenges I have done a fair amount of refactoring. I started off with a simple console app with most of my code in one method. As the number of challenges increased I started moving the code for each challenge into its own class, with methods and properties that were unique to the challenge. I also added a main menu so you can choose which challenge to run. Some of my code to answer the challenges takes some time to run so this was quite a time saver over running each challenge in order.

These challenges are very good at teaching me things about coding. I have got to think about data structures, what is the best way to store this information, classes and methods and return types, how to output data to the screen, how to input/import data.

Before I try and work on the next challenge lets have a look at the first challenge:

Santa is trying to deliver presents in a large apartment building, 
but he can’t find the right floor – 
the directions he got are a little confusing. 
He starts on the ground floor (floor 0) 
and then follows the instructions one character at a time.

An opening parenthesis, (, means he should go up one floor, and a closing parenthesis, ), 
means he should go down one floor.

This is a simple loop through a long string of ( and ) and keep track of the floor number you are on to calculate what floor Santa finishes on. The second part of the challenge asks at what point in your input string does santa reach the basement (floor -1)

Roddenberry comic

Thu, 10 Dec 2015 20:00:45 +0000

The Oatmeal created a comic which you really should check out. It can be found here .

The comic tells the story of a plane crash, in which the 25 year old co-pilot goes back into the burning plane to rescue passengers and lead a group of survivors through the desert to find help.

The co-pilot was named Gene Roddenberry, and after these events he stopped being a pilot and became a writer and eventually created Star Trek.

In these uncertain times when you are stranded in the desert, get out and help someone, it may change the world.

SQL Server Management Studio

Thu, 03 Dec 2015 00:00:00 +0000

I use SQL Server Management Studio all the time for writing queries, getting data and general SQL development.

I have enjoyed seeing the improvements that each new version of SQL Server Management Studio (SSMS) introduced. One great improvement was intellisense.

This feature saves typing and reduces errors by automatically suggesting tables, column names or other database objects.

A common query that I get asked to write is provide a spreadsheet that gives the information that satisfies certain criteria. This is easy to do in SSMS, you can write the query, click execute and the rows that satisfy the criteria are displayed. These rows can then be easily copy/pasted into Excel or other spreadsheets.

A common data item that gets stored in databases is addresses and addresses often contain line breaks to make the data display better. In the earlier versions of SSMS, when you copied and pasted these line breaks were ignored and the data displayed the same in SSMS as it did in Excel. However in the more recent version, theses line breaks got copied across breaking your spreadsheet and making it hard to see what data corresponded with what.

Now I don’t know if this should be described as introducing a bug or fixing one. I can easily argue both sides. If your data contains a line break and you copy this data it should include the line break in the destination, but if it does that it displays badly in Excel.

The fix I have been using until recently is to use the following TSQL command in my queries.

SELECT REPLACE(REPLACE(@str, CHAR(13), "), CHAR(10), ")

This command replaces any line breaks with an empty string. Both Char(10) and Char(13) are needed because you can have different types of line breaks. This is great if you are writing the script from scratch but isn’t great if your are running a stored procedure or your query has a lot of columns.

The answer to this is to use Visual Studio to run your SQL query. In Visual Studio you can write and run queries via Server Explorer and the results produced don’t contain line breaks. I have only just discovered this solution, but so far it has worked and is very easy to do, plus as I do most of my development in Visual Studio anyway it saves me having to open SSMS to test my queries.

Black Friday Deals

Fri, 27 Nov 2015 20:00:45 +0000

Over 50% Off this Black Friday!

Yes your heard it correctly even www.funkysi1701.com has amazing offers.

It is like the whole world has gone “Black Friday” mad, with every company in the world offering amazing and unmissable deals.

So in an attempt to cash in on all the Black Friday fuss I will write a quick blog post about it.

I mentioned 50% off well this month I have blogged 50% less than usual. There is one main reason for this, my new born son James is taking a lot of my time and energy and I think it will take a few months of sporadic blog posts before I get the balance right.

I am not a big fan of spending just for the sake of it. Encouraging people to spend money they don’t have, on things they don’t need shouldn’t be encouraged. However times are hard so companies need to do what they need to, to try and make money.

Star Trek is back (in 2017)

Thu, 12 Nov 2015 20:00:45 +0000

As you have probably heard Star Trek is coming back to TV. I recently joined with the Sci Fi Waffle podcast to discuss.

Sci Fi Waffle: Episode 12 – STAR TREK IS BACK!!

In this episode we discuss the great news that Star Trek IS BACK!!.

Shawn and I are joined again by James Roberts and TrekMates News feed Editor Simon Foster to speculate about the announcement of a new Star Trek TV series due to debut in January 2017.

James can be found on our sister podcast The Battle Bridge and Simon can be found on Twitter @funkysi1701

To listen go http://www.trekmate.org.uk/sci-fi-waffle-episode-11-star-trek-is-back/ , to leave feedback go http://forum.trekmatefamily.com/

How to speak to the whole world

Thu, 05 Nov 2015 20:00:45 +0000

Having a website is an easy way to reach people across the globe. This website has reached people in almost every continent.

I have had visitors in each of the following countries.

United States
United Kingdom
Canada
Germany
Australia
India
France
Vietnam
Netherlands
Poland
China
Russia
Sweden
European Union
New Zealand
Hungary
Norway
Spain
Brazil
Hong Kong SAR China
Belgium
Denmark
Italy
Switzerland
Finland
Turkey
Romania
Singapore
Israel
Indonesia
Serbia
South Africa
Lithuania
Mexico
Ireland
South Korea
Taiwan
Greece
Bulgaria
Ukraine
Japan
Austria
Kenya
Portugal
Czech Republic
Colombia
Philippines
United Arab Emirates
Moldova
Slovenia
Malaysia
Ghana
Argentina
Venezuela
Bangladesh
Iceland
Croatia
Panama
Belarus
Slovakia
Albania
Kazakhstan
Algeria
Puerto Rico
Thailand
Estonia
U.S. Virgin Islands
Pakistan
Latvia
Chile
Sri Lanka
Malta
Costa Rica
Morocco
Ethiopia
Réunion
Uruguay
Honduras
Cyprus
Tunisia
Nepal
Cameroon
Paraguay
Gambia
Peru
Mauritius
Bermuda
Cambodia
Jersey
Egypt
Macedonia

How cool is that! If you know anyone from any country not mentioned above, send them a link to this page and lets see if we can get this blog seen in even more countries.

Only a few years ago it would have been incredibly difficult to communicate your ideas to the whole world, now it is available to everyone at a touch of a button.

The internet has a lot to answer for. On the one hand it is blamed for a decrease of communication between people as they stare at their devices all the time, but on the other hand it has greatly improved communication over larger distances like the use of Skype or the ability to broadcast your ideas on websites and social media.

Top 10 Warp 11 Songs

Thu, 29 Oct 2015 20:00:45 +0000

I don’t want to go to work Just want to hang with Captain Kirk

I am a big Warp 11 fan. Warp 11 are a Star Trek rock band from Sacramento, California. Their songs often feature bad language, sex and drinking but always feature references to Star Trek characters and episodes and a good dose of humour.

I first listened to Warp 11 in about 2004 and have listened to every album they have released since. The second album Red Alert is probably my favourite but all their albums have a few tracks which I love. They have just released their seventh album and to celebrate this I will list my favourite tracks.

All lyrics are by Warp 11 and no infringement is intended, if you like what you see go to their website (www.warp11.com ), or buy their music. Also Warp 11 if you are reading this, come to the UK, I would love to hear you perform live.

It’s Dead Jim

From the fourth album It’s Dead Jim. This song came out after Enterprise was cancelled and is all about the fact there was no more new trek being made. It was a bleak time to be a trekkie and I shared Warp 11’s sadness in this song.

First they took my TOS
Then they took my TNG
Said “bon voyage” to Voyager
Now there’s nothing left for me

Decommissioned DS9
Now there’s no more Enterprise
Erased the animated show
Yeah, she met a quick demise

The final frontier has been finalized
There ain’t no coming back
It’s getting no reprise
It boldly came and went
Got no Trek to represent
I never thought the day
Would come that I had to say
It’s dead Jim

Space is cold and so am I
Just like Kirk my show has died
Everything has gone awry
I guess it’s time to say goodbye

‘Cause Scotty’s gone and so is Bones
They took their final voyage home
And I guess I’ve always known
That some day I’d be alone

Tore down the sets today
Gave the plastic ears away
Sold the props on eBay
Left my life in disarray

It’s dead Jim
Until they bring it back again

7 of Mine

From the second album Red Alert. 7 of 9 is one sexy character so it is no surprise that Warp 11 chose to sing about that in a few songs and I think this is probably the best example.

Flesh and bone and silicone
You’re all by yourself, but you’re never alone
You’re pretty smooth, for a chick full of wires
So cold to the touch
But you set me on fire

Assimilate me
Dominate me
Short circuit my mind
Energize me
Your thighs always rise me
Let’s go out and make your gears grind
Seven of mine

One minute you’re cruel
And then you’re kind
You gave me a virus, but baby I don’t mind
‘Cause I self destruct when I see your behind
I’ll show you the difference between seven and nine

Chorus

Well, life and love are so absurd
And assimilation is the final word
Resistance is futile can’t you see
So, come on now baby and assimilate me
I’ll give you an identity
Locutus ain’t got nothing on me, oh yeah

Let’s grab a shuttle and go to your cube
You bring the oil and honey I’ll bring the lube
It’s not monogamy
‘Cause in the end
When I’m f****** you, I’m f****** all of your friends

Chorus

Old Country Dr.(The Ballad Of Bones)

From the second album Red Alert. A song dedicated to Bones so of course it is great. The use of harmonica makes you think of the country doctor which is a nice touch. This song gets a sequel as well on one of the later albums.

Well, they call me Bones and I’m third in command
Second in line is a man I just can’t stand
He’s a green blooded, pointy-eared, seven year itch
And when he pon farrs
He’ll try to make me his bitch

Well, I may or may not have divorced my wife
But I damn sure pulled the plug on my daddy’s life
And two months later, well, they found a cure
I’m just an old country doctor on a damned five year tour

The damned teleporter
Yeah, it gives me the creeps
The first time I used it
I couldn’t get it up for weeks
Now my tricorder’s busted
And sick bay just stinks
I guess I’ll just go replicate me a drink

Chorus

Well, Jim is the captain of this here ship
We call her the Enterprise, and the crew’s pretty hip
Yeah, the girls all wear go-go boots and miniskirts
And I spend my days patching all the red shirts

Well, I have been asked to do many a thing
Outside the area of my expertise
Damn it Jim, I know I’ve told you before
Damn it Jim, I know I’ve told you before
Damn it Jim, damn it Jim, I told you before

I’m just an old country doctor on a damned five year tour
Just an old country doctor on a damned five year tour

Space Happy

From the second album Red Alert. This song is just solid good fun, with a catchy chorus that I can’t help singing along to.

Gonna take a five year tour
Boldly go where no man’s gone before
Gonna travel to the end
And make new friends

Move ahead warp factor ten
Put a miniskirt on my yeoman
Represent the human race
And we’ll make space
A happy place

To boldly go where no man’s gone before
I think I sang that line once before
(But I’m not too sure)
We’ll be so happy – can’t you see?
A zero-g community

Gonna travel to new worlds
Beam on down and meet some green girls
Gonna see what we can see
A fancy-free galaxy

Chorus

Come and take a tour with me
They’ll be no Kahns or Kobiashi
Gonna travel to the end
And make new friends
With aliens

Chorus

And Thats Why I’m In A Star Trek Band

From the second album Red Alert. Not only does the start of this song make me think of the postman pat theme but it is good fun.

I love gettin’ hummers from chicks dressed as Klingons
Bumpy heads bobbin’ up and down
Like their necks have a spring on
And I know their teeth hurt
But I still get my thing on

And that’s why
And that’s why
I’m in a Star Trek band

I wanna tag team a stripper with Mr. William Shatner
He’ll take the front and hell, I’ll take the back-ner
And when we blow
We’re gonna reach warp factor

And that’s why
And that’s why
I’m in a Star Trek band

Well, people ask me
Where do I find my joy?
Do I like hot girls, green aliens or young boys?
Well, I got two words for you my friend – Leonard Nimoy

And that’s why
And that’s why
I’m in a Star Trek band

And that’s why
And that’s why
We’re in a Star Trek band

Everything I Do (I Do With William Shatner)

From the second album Red Alert. This was probably the first Warp 11 song I heard and it made me want to listen to the rest. Warp 11 don’t just mention trek, they are actual fans as you can tell from the references that litter their songs.

Everything I do, I do with William Shatner
Loosing all my hair while my belly’s growing fat-ner
And when I drink too much
He holds my hair in his hands
And while I’m blowing chunks
He treats me like the captain

Oh, when we get together we boldly go
Talk about our favorite episodes
I don’t want to go to work
Just want to hang with Captain Kirk
And if I had my way I’d make it so

Everything I do, I do with William Shatner
We drink a couple of brews
Go to the Laundromat-ner
When our clothes are dry
We fold them up together
‘Cause everything I do, I do with William Shatner

Chorus

Hey, spend more time with him than George Takei
Say full speed ahead and we’re on our way
He is so mysterious
His middle name’s Tiberius
And he’ll make me his cabin boy someday

Hey Bill – Let’s go get a tan
Hey Bill – And a green woman
Hey Bill – We’ll kill some Romulans
Hey Bill – I’m your number one fan

‘Cause everything I do, I do with William Shatner
We go to the rodeo and put on our cowboy hat-ners
When we paint the town
We buy each other dinner
We’re fully functional
Just like Brent Spiner

Everything I do, I do with William Shatner
And don’t you wish that you could do it too

What would William Shatner do?

From the fifth album I Don’t Want to Go to Heaven as Long as They Have Vulcans in Hell. Everyone’s been in situations where they would like to keep there fandom a secret. And what would William Shatner do?

I’ve got a girl, the coolest girl in the world
But she ain’t fly for the guys in sci-fi
And I’m a guy with a model Enterprise
Imagine her surprise if she ever finds out

I’ve got a queen, but she only wears blue jeans
And I am a guy in Kirk t-shirt
Black boots and a red miniskirt
I’ll be in world of hurt if she ever finds out

I’ve got to think about it
I’ve got to work it through
If she learns my dark secret
What would William Shatner do?

I’ve got a chick, in bed a lunatic
Won’t be my whore if she knows I love Star Wars
And I know only one thing for sure
I’ll never score If she ever finds out

I’ve got to make a choice now
Deceive her or tell her true
I know Vulcans never lie but
What would William Shatner do?

I love War of the Worlds, got a b**** for green girls
Alien nations, deep space stations
I love flying saucers, say “live long and prosper”
“May the force be with you”, Mr. Sulu

I want to be your man, “by your command”
Don’t want us to be undone, “danger Will Robinson!”
And if I decree “beedee beedee bee”
Does that mean we’ll be through?
What would William Shatner do?

Trekie Girl

From the second album Red Alert. When I first listened to this song I was a single young man, so of course I dreamed about finding my own Trekie Girl. My wife now shares my love of Trek and looks extremely hot in a starfleet uniform.

She knows every episode
She wears the boots that go-go-go
She’s got a red miniskirt
She makes me play Captain Kirk

Trekkie girl, Trekkie girl
We go to conventions
She gives me erections
Girl – my Trekkie girl
She paints her skin green
And it’s not Halloween
She rocks my brave new world

She always acts so logical
She drives a Volvo Iike it’s our space shuttle (of love)
She looks just like Britney Spears
Just add a few pounds and some pointy ears

Trekkie girl, Trekkie girl
She loves outer space
Has a black and white face
Oh, girl – my Trekkie girl
I’m her Leonard Nimoy
She’s the real McCoy
She’s got a Spock haircut with curls

And on some nights she tells me
Put my symbiote inside her
She separates my saucer
Yeah, she knows just what to say
And on nights when I loose power
Well it doesn’t even phaser
She grabs my captain’s log
Torpedoes away

She knows every episode
She wears the boots that go-go-go
She’s got a red miniskirt
She makes me play Captain Kirk

Trekkie girl, Trekkie girl
We go to conventions
She gives me erections
Girl – my Trekkie girl
She paints her skin green
And it’s not Halloween

Oh, girl – my Trekkie girl
She loves outer space
Has a black and white face
Oh, girl – my Trekkie girl
I’m her Leonard Nimoy
She’s the real McCoy
My girl – Trekkie girl

A Song for People Who Never Watch Star Trek

From the third album Boldly go down on me. This really tells you all you need to know about original series trek, plus I like singing the first line of this song to my son (also called James)

There was this guy
His name was Jim
You can call him Kirk
Or the Captain
He had a tan
And lots of women
Every episode
He got to blow his l***

He worked with Spock
He was a rock
No emotion
Nothing could rattle him
His blood was green
And Pon Farr could make him scream
And lose control
Though that’s illogical

Each week they’d all warp through space
Fight an alien race
And the Vulcan would pinch a neck

If you want to learn more
Go to the video store
Or just listen to this
Song for people who never watch Star Trek

Then there was Bones
Dr. McCoy
Worked with Nurse Chapel
And removed the red shirt shrapnel

Chased the miniskirts
Screaming, “Dammit, Kirk!”
Wait that wasn’t him
He was screaming, “Dammit Jim!”

Each week on the Enterprise
All the red shirts would die
But no one noticed or got upset

And the next generation
Will watch in syndication
Or just listen to this
Song for people who never watch Star Trek

And for three seasons
You know what they’d do
They’d boldly go
With the rest of their crew

Like Scotty
He was a Scot
And Chekov
He screamed a lot
And Uhura
And Sulu
But let’s face it
After Kirk, Spock and McCoy
Who gives a f*** about the rest of the crew?

The Saddest Song Ever Written About Star Trek

From the fourth album It’s Dead Jim. This song really encapsulates the sadness of Star Trek’s cancellation and the passing away of some of Star Treks greats.

It was a warm summer’s night when I sat down to write
The prettiest song ever written about Star Trek

The sky was clear
Full of stars above
I thought of Kirk and his crew
And their mission of love
And I sang the first verse
Of the prettiest song ever written about Star Trek

When the words came out
Everything was wrong
For those who’d boldly go
Had already gone
‘Cause the show was cancelled
And the dream was dead
And a red alert started wailing in my head
And I realized I had written
The saddest song ever written
About Star Trek

Now the sky was dark
Devoid of love
And Roddenberry’s ashes were floating up above
Raining down on a man
Singing the saddest song ever written about Star Trek

And now Bones is nothing but bones
He finally went and took his final voyage home
And Scotty’s body just couldn’t take much more
And I’m sure this is the saddest
Song ever written about Star Trek

I thought about space
And the final frontier
And I shed a tear as I finished
The saddest song ever written about Star Trek

James goes on an adventure with Daddy

Thu, 22 Oct 2015 20:00:45 +0000

This weekend I took James on an adventure. As he is only 8 weeks old it was a fairly low key adventure but hopefully he enjoyed himself.

St Michael le Belfrey have been organizing an event for Dads once a month and this month I wanted to give it a go. I have been looking forward to going for a while.

Looking after a baby is woman’s work. Now I am not being sexist but men are just not equipped. We can’t feed them, we don’t form the same emotional bonds and I suspect I am sleeping through the worst of his crying.

That said I still want to help and be involved in looking after my son. But my natural inclination is to let mummy do most things and help where needed. Going to this dads event was my first chance to do things without mummy being around.

So after James had his first feed and his bag had been filled with bottles of milk, nappies, change of clothes and all the other essential accessories for looking after a baby, I left the house with him strapped to my chest.

The event was very low key. Once I got there I unloaded James from my chest and was left to spend time with him with the occasional chat from other dads, offers of coffee and sausage sandwiches from the organizers.

James loves staring at the ceiling and the Storehouse had lots of bright lights for him to stare at. He also got to say hello to one of the other babies he has met before at church and the mums group my wife goes to. But it was not all smooth sailing, James didn’t like me sitting down, so I had to stand up holding him close to keep him calm. If I tried to sit down and offer him his bottle he would get grumpy.

I think it is great that there are events like this for parents. A lot of events are targeted at mothers which is as it should be as they have the majority of work to do, but dads need the odd event to support them as well. Hopefully me and James will be back for more adventures like this.

One of the things I am looking forward to as James gets older is taking him to places and spending time 1 to 1 with him like I did with my father. I don’t know what his interests will be but whatever it is I want to be there to encourage him.

Backing up SQL databases to Azure

Thu, 15 Oct 2015 20:00:45 +0000

I recently read a blog post by Pinal Dave about how you can backup straight to Azure Storage. The procedure he described is only available for SQL Server 2014 or later.

I won’t go into detail of this method as Pinal describes it better than I can, but the basic of it requires setting up credentials and then running a backup command that includes the URL of the storage container on Azure.

Unfortunately I am running SQL Server 2005 so this process will not work for me but it did start me thinking of what ways there might be for me to use.

The next thing I tried was Microsoft SQL Server Backup to Microsoft Azure Tool . Unfortunately I did not get this tool to work correctly on my setup. However it sounds like a flexible tool that allows compression and encryption of your backup files. This tool redirects your backup files to your Azure Storage so even if I had got it to work correctly it would not have been an ideal solution as I want local copies of my backup files as well.

After this I started to look at powershell again. Following on from my recent success with powershell I know how to connect to my Azure account so all I needed to script was copying a file from my server to Azure.

Get-ChildItem *.bak -File -Recurse | Set-AzureStorageBlobContent -Container $DestContainer -Force

This command gets all the backup files in a directory (the recurse switch looks in sub directories as well) and then pipes them to the Set-AzureStorageBlobContent command. This command uploads them to the storage container defined in $DestContainer. I have added the Force switch so that it will replace any files on Azure which have the same name.

I have only been using this script for the last few days but so far it has been working well. Now if I completely loose all data from the office I can restore from any other location using the data saved on Azure. A great improvement to my disaster recovery policy.

The Hunted, TNG S3 E11, The Battle Bridge

Thu, 08 Oct 2015 20:00:45 +0000

My latest podcast can be found http://www.trekmate.org.uk/the-hunted-tng-s3-e11-the-battle-bridge/ and feedback can be left on the trekmate forums http://forum.trekmatefamily.com/2015/10/the-hunted-tng-s3-e11-review-the-battle-bridge/

While the Enterprise-D is reviewing a seemingly idyllic planet’s application for Federation membership, an escaped prisoner leads its crew to discover an ugly secret: the government’s shameful treatment of its war veterans.

Today James and Lou discuss (The Hunted) and what we think of it and much more as well as listener feedback with tonight’s guest host Simon.

Credit: Main Title Theme (TV Edit) was arranged by Dennis McCarthy and composed by Jerry Goldsmith and Alexander Courage

You can download the MP3 version of this show by clicking here .

Copying settings to an Azure Website

Thu, 01 Oct 2015 20:00:45 +0000

The Software as a Service (SaaS) website that I work on has been sold to lots of clients now. Which is great news.

However the more Azure websites we have, the more websites we have to administer, especially if like us you take advantage of Traffic Manager which requires multiple website in different regions. Azure has some great options for making this administration easier. One job is adding all the settings onto the Azure portal, so far I have been manually adding these, but a quicker way is to use powershell.

PowerShell

PowerShell is everywhere these days. You can use it to control Servers, Active Directory and Exchange. So it is no surprise that you can use it to control Azure.

Open a powershell window and run the following command.

Get-AzurePublishSettingsFile

This command opens an IE window which you can login to Azure and download a file which contains settings that Azure can use. Save the *.publishsettings file and run the following command.

Import-AzurePublishSettingsFile "C:\MyPublishSettings\mysubscriptions.publishsettings"

This imports your Azure settings so that PowerShell can do clever things.

Select-AzureSubscription -Default "mysubscription"

This selects which of your Azure subscriptions to use. Now run the following to import settings into PowerShell.

$s = @{"DebugEmailAccount"="test@example.com";"SiteWarningBannerText"=""}

And finally run the following to import this settings into the Azure website you specified.

Set-AzureWebsite azure-websitename -AppSettings $s

Sounds easy doesn’t it. Well it is. The hardest part is getting the settings in the correct format to be imported but this is only string manipulation.

For my project I already have a build script which populates a settings.config file with all these settings, so I have just duplicated this to create a settings.config.importtoAzure file. Next time I have a website to create, I can create it on Azure and run the above script, pasting in the settings file that my build has already produced for me.

This only scratches the surface of what you can do with Azure and PowerShell, hopefully I will do far more in the future.

Adding Internet Connection Resiliency

Thu, 24 Sep 2015 20:00:45 +0000

I have been asked to investigate adding a fail over internet connection to one of our offices.

Currently this office is connected via a wireless link with our head office. Unfortunately we have recently experienced some poor performance with this connection, this has now been corrected but like all technology there is the chance of it failing in some way in the future and causing loss of business because of it.

Internet Connection

This is one of my first considerations. I need a reliable internet connection and there are a lot of options to consider, lets look at a few.

ADSL Connection This is your standard internet connection that most homes have, you get around 20Mb/s download but only 2Mb/s upload for a fairly low price per month. Speeds are dependent on area and the quality and distance from your exchange, some rural areas have speeds much lower than this. As this connection is for an office I want the best connection I can afford so would rather not go down this option if I can avoid it.

FTTC (Fibre to the Cabinet) This is what ISPs typically mean when they say superfast broadband. The cables that have been laid to your exchange cabinet have been upgraded to fibre optic cables which are capable of much faster transfer speeds (eg 40Mb/s download and 10Mb/s upload). This is my ideal choice, however BT Openreach are responsible for upgrading the country to these cables and they are not very quick about it. A few years back when investigating this FTTC had almost reached our office, but when I investigated today nothing seems to have progressed. Both FTTC and ADSL require a phone line into the office to run on.

Leased Line This is where your ISP connects a dedicated line to your office which provides you a much faster connection (upload and download speeds are the same) but this is very expensive. We have this for our head office as this is where our servers are located and the faster speed benefits the entire company. We do not want to do this for every branch office as it will be too expensive.

Spur from a Leased Line It could be possible to branch or spur off the leased line to our second office. This would be one leased line that terminates in two locations instead of one. However due to the distances involved I do not think this will be possible in my case and may end up costing as much as a second leased line.

ISP As well as the type of connection I need to decide who will provide it. There are hundreds of ISPs out there but I like to use ISPs that have a good reputation or I have used in the past and have been happy with. I do not stick with one ISP for all my connections as I like to have some connections that continue to work should that ISP be experiencing problems. ISPs that I would recommend are YDS, Eclipse and PlusNet

Money Off While I am talking about internet connections the government has been offering a scheme offering businesses money off upgrading their connections to a superfast connection (either FTTC or Leased Line) More information can be found on your councils website, If you have an office in York have a look here for more details. We got £3000 off installation so worth investigating if you qualify.

VPN Connection

Offices require access to far more resources than just the internet. This particular office requires access to our email and database servers as a bare minimum. One way to access a remote offices network resources is through a VPN (Virtual Private Network) connection. So my next consideration is how to establish a VPN link between our offices.

Software Windows Server includes RRAS (Routing and Remote Access Service) which you can use to configure a VPN connection. One important thing to note is that the server needs at least two network connections, one on the internet with a public IP and one on the internal network.

Hardware The more expensive routers often have settings that allow the configuration of a VPN connection.

I have used both of these methods in the past. RRAS is a pain to work with but has evolved with the newer versions of windows server, so may not be as bad as I remember. Using a router to do VPN introduces its own set of problems and there is no guarantee of avoiding RRAS, you may still need it to authenticate the VPN provided by your router.

Multiple Connection Handling

Internet connectivity is being provided in two ways, the primary way is via a Wireless link and the second way is via one of the options above.

There are lots of options to deal with multiple connections, do I want to load balance both connections and use them all the time, do I want to fail over onto the second connection only if the first one fails, or do I want it to be a manual process or switching over to the backup network if problems occur.

As you can see there are lots of different technologies to consider before I can add resiliency to this office and I haven’t even started to think about if an additional server will be required.

Unmanaged Database Deployment

Thu, 10 Sep 2015 20:00:45 +0000

For the past few months I have been deploying changes to my companies database every couple of weeks or so. Over a weekend when the database was not being used I would make a backup of the database, load up visual studio, deploy the database changes and copy a couple of front-end files.

My weekends and evenings have recently become a bit more precious to me since the arrival of my son James. At suitable times to run the database upgrade I am either, asleep, about to fall asleep or be spending time with James.

This entire deployment can be set to run at a time of my choosing using SQL Server Jobs, (not sure why I never thought of doing this before). I still need to check everything is working after it has run. The job can send me an email if it encounters any problems so I can still ensure the company has the minimum of problems but has more frequent changes.

Unmanaged Database Deployment

OK so what do I need to do to set this up.

Test all your code changes are working correctly on a backup of the database and everything is committed to source control.
Backup everything that is going to be changed so it can be rolled back in case of a problem. I don’t rely on the daily backup jobs for this, I do my own. Maybe I am a bit paranoid, or maybe I am just being cautious.

BACKUP DATABASE DBName TO DISK='E:\SQL Backups\Filename.bak'

I would include in the backup file name the date and time of the deployment for easy reference later on. For the front-end files these are backed up daily and only ever change during a deployment so I am going to rely on the day to day backups, plus they are in source control so in a worse case scenario I can look there.

From Visual Studio create a deployment script using the publish option and save as a file on your database server.
Create a SQL Server Job and give it a descriptive name.
In steps create a step called backup and enter the T-SQL code above. Set this to stop on failure and continue on success.
Create a second step called upgrade of type operating system with the following code.

sqlcmd -S (local) -i "E:\SQL Backups\deploy_test.sql"

The -S parameter is the name of your SQL Server instance and the -i parameter is the path to the sql script you generated from Visual Studio above. Set this job to quit reporting success or failure.

In schedule create a date time you want the job to run, make sure this is a one time job as you don’t want it to try and run again possibly causing problems.
In notifications I have set to email on completion, you can set it to email on failure only. I would prefer to know the outcome of the job regardless.
Using SQL Server jobs to copy files over the network is possible but is not trivial to setup so I am going to use windows task scheduler for this.
Create a job in task scheduler to copy all the ADPs or ADEs that have been changed. If the SQL job fails you will need to copy the old versions of these back but that is simple enough to do.

This process can be expanded to check code out of source control and do even more automated deployments but for now this is good enough for my purposes.

Baby Magic and Becoming a Father

Thu, 03 Sep 2015 20:00:45 +0000

I am pleased to announce the birth of James David Martin Foster weighing 9 lb 12 on 19th August 2015.

Mother and Baby (and Father) are doing well.

I am not a baby person, I hate children and I run a mile from other people’s babies.

But James performed some magic on me and now I will do anything for him. If you have had children you probably know what I am talking about and if you haven’t you are probably like me 24 hours before James arrived.

During the pregnancy I had been living in denial that I would become a father, despite the wife speaking about nothing else and our home filing with baby equipment. It was only minutes before I first saw my son that I changed.

Due to James’s large size, the birth wasn’t easy. After about an hour of pushing the midwife called in a team of doctors and nurses to help and I think it was at this point that it started to hit me. I am not an emotional man, if anything I tend to repress my feelings but I completely and utterly failed to do that on this occasion. By the time the baby had been placed on my wife’s chest tears of joy were streaming down my face and this continued for some time. Moments after birth James had a tight grip on my finger. His tiny hands were like miniature versions of my own hands.

I have been asked what it felt like holding James in my arms for the first time, this happened on the following day. I am not sure if I can put it into words, but a few adjectives come to mind, amazing, great, fantastic, humbling. James depends on me (and his mother) for everything and his existence has put a new spin on everything I do. I want to be there for him, help him learn stuff, provide for him and love him.

I have been working very hard recently, spending my free time writing this blog, learning stuff and doing other work. The reasons for this hard work are now clear, I work so I can provide for James and so I can spend my free time with him.

Before James arrived I had it in my head that his mother would do the majority of the caring for him. This is not going to happen now that I have met James. I want to cuddle him, change his nappies, dress him, hold his hand, take him to school, watch TV with him, stroke his head when he is ill, tell him about my life experiences, introduce him to wine, tell him about girls, help him move home, etc.

I am not saying being a parent is going to be easy. I am pretty clueless about what is in store for us, but already I am worrying if he is OK, trying to calm him down when he is upset and being woken in the early hours.

Anyway only time will tell if I continue being this positive about fatherhood, but so far I am loving it. Normal blog posts will hopefully resume over the next few weeks (or when I have time).

Due Dates

Mon, 17 Aug 2015 20:00:45 +0000

As I write this I have less than 2 days until the due date for the birth of my first child.

Its a very exciting time, but it is also infuriating waiting for it to arrive. Even in the 21st century science can not predict with any accuracy when a baby is going to arrive and we just have to wait for mother nature to do its thing.

I feel like I am stuck in limbo at the moment I have lots of work to do, lots of development changes to do on the databases, but I also know that at a moments notice I need to drop what I am doing and try and be a father.

Due dates for babies is based on average pregnancies lasting 40 weeks, but there is no such thing as an average pregnancy. But to keep this blog post on topic I somehow need to compare due dates for pregnancy with software development.

In software development we often get asked for estimates on when a particular development task will be finished. Estimates in software development are very difficult to make accurate as there are so many variables that you do not know the answer to until you start getting stuck into doing the work. One technique is to try and split the work down into small steps and estimate how long each small step will take and then add them all together. But even this technique still has a large margin of error.

What would it be like if when management asked we gave an estimate similar to what pregnant women receive? Instead of a deadline when all work has to be finished by, you have a date when there is a 50% chance of the work being finished.

I think management would have a difficult time dealing with this amount of uncertainty, I certainly am waiting for this baby to arrive.

How complex are my stored procedures?

Thu, 13 Aug 2015 20:00:45 +0000

Recently me and my development team have been looking at performance and how to improve it.

One area of improvement we have identified is with our stored procedures, which we plan to rewrite. But how do we identify which are the easiest to rewrite and which are the hardest?

My first thought was maybe to count the number of times each keyword is used and rank them somehow. eg each JOIN gets 5, OUTER APPLY gets 20, each term in WHERE gets 1, and then combine that with the length of the query and how many parameters.

However doing a bit of googling I came across the following blog and sql script . This script analyses the stored procedures in terms of number of lines of code, number of parameters and number of dependencies.

The complexity is divided into SIMPLE, MEDIUM and COMPLEX.

WHEN NumberOfLines * NumberOfDependencies * NumberOfParameters < 5000 THEN ‘Simple’
WHEN NumberOfLines * NumberOfDependencies * NumberOfParameters < 10000 THEN ‘Medium’
ELSE ‘Complex’

This gives a fairly good estimate of which stored procedures are the most complex and which would probably take the longest time to rewrite.

ICYMI What does it mean?

Mon, 10 Aug 2015 20:00:45 +0000

For a while now I have seen ICYMI splattered around the internet. I had no idea what it meant so just ignored it but I have finally had enough and I googled for a definition and I now know what it means.

ICYMI – In Case You Missed It

ICYMI is often used on social media sites to indicate articles that you might have missed.

Software Development is littered with acronyms and technical terms that I don’t always know the meaning of.

When reading it is easy to ignore them and try and understand the rest of the article. However there are loads of resources out there that can explain technical terms in a wide variety of different ways.

I have tried to explain a few different terms and concepts on this blog . In the process of writing them I often do a bit of research and read a variety of different definitions, some confuse the heck out of me, while others make me go ahh that makes sense now.

When reading to learn, don’t be afraid to dig deeper and understand it all, don’t read the same terms again and again not understanding them like I did with ICYMI.

Common git commands

Thu, 06 Aug 2015 20:00:45 +0000

I use git as my source control system. Here are some of the most common git commands in no particular order.

Check Out a git Repository git clone /path/to/repository
Add files to commit git add
Commit files to git git commit -m “Commit message”
Push changes to server git push origin master
Show status git status
Create new branch git checkout -b
Switch to a branch git checkout
Get remote changes git pull
Merge a different branch into the current one git merge
View merge conflicts git diff
Temporarily stash uncommitted changes git stash
Undo a commit git reset –hard
Show details about a commit git show
Show version history of current branch git log
Gets all remote branches git fetch origin

Most of the time I use the git extensions tools to do my git work, but there are times when only the command line will do.

Volunteering for a Technical Team

Mon, 03 Aug 2015 20:00:45 +0000

For the past few years I have volunteered on the technical team at my local church St Michael le Belfrey .

For the church to function well it requires volunteers in all sorts of roles, people to welcome visitors, people to pray, people to play music, people to look after children and many other roles.

My natural ability lies more with technology than with people so when I decided that I wanted to get more involved it was the technical team that I volunteered for. The technical team is split into two main parts, the PA team which concerns itself with ensuring everyone can hear everything OK and the Projection team which concerns itself with everything that gets projected. I have no knowledge about what sounds good and what doesn’t so it was the projection team that I joined and have been involved with.

The church has three projectors, one big one and two slightly smaller ones for the side aisles, and a PC located at the back of church which is what the projectionist uses to control what gets projected. There are many things that get projected, lyrics to songs, PowerPoint presentations and the occasional video and to make things easy one piece of software is used to control all of this EasyWorship .

EasyWorship contains a huge database of song lyrics, so you can quickly select what ever is being sung and project the lyrics on screen. It also allows the addition of PowerPoint files, image or video files and even a web page and they can be projected. EasyWorship allows different things to be queued up in order and that order can be saved as a schedule.

The projectionist’s job is to follow along with the order of service and project things onto screen at the right time. Simple! No not really. The worship leader will skip songs, play songs in a different order, add new songs, videos will fail to play, easy worship will freeze up or crash or the liturgy that the vicar wants isn’t in the EasyWorship database.

You need to be on your toes as a projectionist but over the years I have become a good projectionist and have really enjoyed my time doing this. You can see me having a laugh with the Head of Music and Worship on the right.

More recently I have taken on a bigger role for the projection team. I have been coordinating the rota for the volunteers on the projection team. This has given me new challenges, but it has been great to work closely with the Technical Manager and more recently the Head of Music and Worship and get to know the whole team better. The biggest problem that the team has is insufficient volunteers, it had this problem when I joined the team and it has this problem today. If you are reading this consider if this is something you could get involved with? The team has some really hard working people who do an excellent job each week, just wish we had a few more to ease everyone’s work load.

I am taking a break from the projection team as my wife is due to give birth to my first child in the next few days. I have had a great time, I have worked with some great people and for the most part I will miss volunteering. There are of course things that have not been great, volunteers that have left the team, changes at the church, and many early mornings when I would rather still be in bed but I am glad that I have been involved.

Upper Pylon 2 – 1 x 09: The Passenger

Thu, 30 Jul 2015 20:00:45 +0000

I recorded another podcast the other week it can be found http://www.trekmate.org.uk/upper-pylon-2-1-x-09-the-passenger/ and feedback about it can be left on the trekmate forums http://forum.trekmatefamily.com/2015/07/thoughts-on-up2s-the-passenger-or-vantika-x-33/

This week on Upper Pylon 2, we are joined by our first UK guest, Simon (aka funkysi1701 ) as we take a look at the Bashir-centric “mystery” episode, “The Passenger.”

When a former prisioner, Rao Vantika (a name you’ll hear COUNTLESS times throughout the episode) is killed, his pursuer believes that the late murderer is not really dead, and continues her investigation at DS9 while an important shipment is scheduled to arrive. Will Vantika reappear, and if so, who will he appear as? Well, if you’ve seen the opening of the episode, it really isn’t all that hard to deduce. But join the UP2 crew as we dissect this little caper.

As always, be sure to leave us your feedback about the episode!

10 facts about Windows 10

Wed, 29 Jul 2015 20:00:45 +0000

Today Microsoft release Windows 10 so what else can I talk about? I like many technology people are very excited today. This version of window is the first that Microsoft have release for free.

10 You heard that correct Windows 10 is free. If you upgrade your device to windows 10 before 29th July 2016 it is free and will continue working on that device forever.

Upgrade to Windows 10 within the first year and it is yours free, forever. 
No subscriptions, no additional costs.

9 As with any upgrade always backup your computer and any important files before you do anything. Microsoft and any IT expert will tell you this. For a great guide to upgrading see Scott Hansleman’s blog . I have upgraded my laptop without any problems but there will always be exceptions so take extra care.

8 Windows 10 is the LAST version of windows. I don’t know what this means but I can take a guess.

Typically Microsoft has created a new version of windows every few years. With every new version there has come headaches and problems as people try to make software and hardware work with the next version. Microsoft is instead going to try and release updates and patches to the operating system to keep things up to date.

However I can see that only working for so long. Eventually Microsoft will make something new but it is unlikely to be called Windows. Maybe it will be something hosted in the Cloud, as they have a really strong presence with Azure which is only going to grow as time goes on.

7 Windows 10 is available in 7 different flavours.

Windows 10 Home – equivalent to the Home editions we have had since XP. This version is designed for your typical consumer.
Windows 10 Mobile – this version is designed to run on phones and tablets. Unlike previous Phone versions, everything you can find on the desktop OS should be available on the phone (but optimised for the smaller screens)
Windows 10 Pro – this is the version I will be testing with my work PCs. It is designed for small businesses and allows domain joining.
Windows 10 Enterprise – this version is only available via Volume Licensing so not easily available to the typical user but is aimed at larger businesses.
Windows 10 Education – another version that is only available via Volume Licensing but targeted at schools and universities.
Windows 10 Mobile Enterprise – this version is for mobile and tablets in the business world. Standard policies can be applied to all your employees mobile devices.
Windows 10 IoT Core – this is an interesting version designed to run on hardware and Internet of Things (IoT) devices. I would quite like to play with this version on my Raspberry Pi

6 Windows 10 introduces a brand new web browser called Edge. For years web developers have hated Internet Explorer, for its lack of support for the latest web standards. Internet Explorer 6 caused many users to migrate away to browsers like firefox and later Chrome and despite Microsoft’s best effort with the later versions people have not moved back. Edge is an attempt at a fresh start.

5 In 2011 Apple introduced the personal assistant Siri, well now Microsoft are bringing us Cortana the windows equivalent. With Cortana you can ask your computer to do anything, set reminders, search the internet, answer questions, anything. With advanced voice recognition you can speak as well as type your questions. Cortana is now part of Windows 10.

4 Windows 10 brings back the start menu. The start menu you have always used to access programs works the same but features of the windows 8 start screen have been incorporated into it.

3 Windows 10 is more secure. Security patches are frequently released for all operating systems, but windows 10 has been designed with security in mind. For businesses there are options to completely lock down your users environment only allowing ‘white listed’ programs to run.

2 Continuum is one of the most revolutionary things about Windows 10. Continuum is the notion that one computing experience is continued regardless of the device used or the location. Continuum will automatically adjust your screen to make the best use of its size. Tablets and phones have the tiled view and start screen, if you connect a keyboard and monitor it will behave like a full blown PC. This is mind bending, that a phone running windows 10 can do as much as your laptop.

1 Today I am off to a Windows 10 launch event where I expect many of these features will be discussed and most likely many more.

Writing better Git commit messages

Sat, 25 Jul 2015 20:00:45 +0000

I always use source control for my coding changes, however some of my commit messages leave something to be desired.

I always try to write a commit message but I often think that the change themselves should be enough to indicate what I did. I also don’t need to include who made the change or the time and date and that gets included automatically.

Here are some tips I have found that may help me in the future.

Ask yourself why you are making this change. The Who, When and What are already being covered so it is only the why that needs including in the commit message.
If your commit breaks something or causes side affects or a new dependency this should be included in the commit message. If your commit breaks functionality, consider if you really need to commit it yet, maybe only commit once fixed?
If your commit includes a long list of changes consider if the commit needs splitting into several commits. It is easy to only commit one or two files write a specific commit message and then commit the rest of the changes separately.
Consider including a subject for larger commits. The following comes from the git manual.

Though not required, it’s a good idea to begin the commit message with a single short 
(less than 50 character) line summarizing the change, followed by a blank line and then 
a more thorough description. The text up to the first blank line in a commit message 
is treated as the commit title, and that title is used throughout Git.

If you use a subject follow the following conventions:

Limit of 50 characters
Start with a capital letter
Do not end with a full stop
Use the imperative mood i.e. write as if issuing a command

My Love/Hate Relationship with Microsoft Access

Fri, 24 Jul 2015 20:00:45 +0000

For historical reasons I use Microsoft Access as the front-end for all my line of business databases.

Access allows you to create an ADP file that can connect to your SQL Server database. However Office has dropped support for this, so the last version of office that I can use is Office 2010.

Before I start hating on Access there are some good things about the platform. Users with very little knowledge can create a database fairly quickly. Development time is also fairly low as it doesn’t take long to create a new form, and add a few buttons which open different reports.

For the last 5 years or so I have built a range of different forms in Access, (and even if I do say so myself), I have got quite good at creating forms that users can be more productive with. Access has the concept of a subform which is a form embedded in a form, each form can be connected to a different query and can be connected to each other so it is relatively easy to create something quite powerful.

Access report are an easy way to create professional looking reports that can be sent out to clients, or used by management and they also have the concept of subreports, (same as subforms but embedded in a report), this means you don’t have to write one query to pull back every piece of data that shows up in a report.

VBA or Visual Basic for Access allows you easily do things programmatically, like run something if something is selected or a button is clicked. This is not the easiest programming language to work with, however once you have used it for a while you get used to its quirks. My favourite function is IIF, the syntax is fairly simple IIF(expr, truepart, falsepart), if expr evaluates to true the truepart is executed (executed can be as simple as displaying certain text in certain conditions), if it is false the falsepart is executed, and you can also chain them together which I have done to produce something very complex looking.

Now for the negative side of things. Access creates a binary ADP file, binary files can not be version controlled in git or subversion. This is a big problem as I want to keep track of every change I make. It also means that branching or working with others on big projects is also difficult.

One work around I have found to this problem is AccessSVN . This program takes an ADP and creates text files of every form and report. It also can create an ADP from these files as well, however I have had varying success with this feature so I prefer to just create the text files if possible. What I do after I have made a change to the ADP is to run AccessSVN, create text files and commit these to source control. You can then see what controls have been changed or added. It also helps with bugs as I have found that sometimes while testing access will apply a filter to a form, which will break things if deployed, which is very annoying.

Access can only do what access was designed to do. There are some advanced features that we would like to add to our databases. Things like sending emails or uploading reports to a website. This is not Access fault as it was never designed to be able to do such things but I am often asked about what is possible.

End of support for Office 2010 is approaching and I have plans to stop using Access but until then I need to keep supporting our existing front ends.

7 Warning Signs that you’re a Bad Programmer

Thu, 23 Jul 2015 20:00:45 +0000

Am I a good programmer or a bad programmer?

Here are 7 signs that might mean you are more towards the bad end of the scale.

You fail to plan before you start coding

I am very guilty of this, I love to jump into the code before thinking. But I am slowly starting to see the advantages of sketching a few things out before I write any code.

Most of my coding recently has been SQL based and I am starting to like sketching out table designs.

You fail to use version control

I have blogged about using version control in the past but it really is a very useful technology. It not only keeps track of every file in your solution and stores its entire history, but you can separate different versions into branches and see who changed what and when (and if the commit message is detailed enough why).

You use bad variable names

Whats wrong with variable1 and variable2 as variable names? Variables should be named after what they do or what they contain, but it is very easy to start copy and pasting them and before you know it you have a variable1 and variable2 situation. Remember that Visual Studio has some powerful refactoring tools so relatively easy to get them back in order.

You repeat your code

I’ve been reading Pragmatic Programmer and one of their first tips was DRY – Don’t Repeat Yourself. They go to extremes with nothing being repeated anywhere in your code, but I don’t think you need to go too far, you can be reasonable. If you have the same code repeated four times, then creating a function for this code will dramatically improve your code.

You struggle to understand your own code

Why did I do it this way? I feel like I am always trying to remember the reasons I chose to code the way I did previously. Part of the reason for this is that I am constantly learning at the moment, but there is more that I can do to help myself understand in the future.

You are selfish and don’t share

I am not selfish, if I learn something really cool I have to tell someone. I like writing stuff on here to share with the world. I am thinking of news ways to train my staff.

You work on multiple projects

Yes Guilty as charged with this one. I have multiple projects that I am working on at work, I also have a fair few personal projects that I would like to work on. Not sure I completely agree that multiple projects equals bad programmer.

Things to know before working on your database

Mon, 20 Jul 2015 20:00:45 +0000

I recently saw this blog post by Brent Ozar that I thought I might discuss.

Brent listed 13 questions to ask about a database before you start working with it. I am going to go through these 13 questions and expand on them based on my experiences.

Is this database in production now?

I think it should go without saying that the first thing you should find out is if your database is in production. If its not in production you can do what you like and no one will notice. I know what databases are in production and which aren’t where I work so I can answer this one.

If this goes down, what apps go down with it?

What apps are running on what databases is a good second question. I have at least one database which has multiple front end apps. At first glance you would think that that these two apps are not connected but they are, I need to be careful with both of these apps to make sure they don’t break each other. I know what apps run off what databases.

When those apps go down, is there potential for loss of life or money?

This is a difficult question so I will split it into two. Loss of life, my databases don’t control life support machines or nuclear weapons so my first instinct would be to say no. However it is not that straightforward, what if your app allowed contractors to know the location of dangers inside a property they were working in. Once your app goes down, they could have an accident due to lack of information. Loss of money, this one is more straightforward. Time is money in the business world so any time that your app is down and your employees are not able to work is a loss of money. If your database is linked to an eCommerce site, the loss of money could be extremely high. I know what affect downtime will have on my users and business.

How sensitive are these apps to temporary slowdowns?

Similar to the previous question, a slowdown can be as serious as downtime for some applications. Luckily most of my apps are internal only so are not seriously affected by slowness.

When was the last successful backup?

I manage the backup schedule for all my databases so I know exactly when each one was last backed up. When ever I do anything to a production database I will run a backup so I can roll back in case of problems. As part of developing changes I run my changes on a backup of the data. I can script all my changes and repeatedly run them against a backup until I am sure no problems will occur.

When was the last successful restore test?

More important than a backup is testing restoring your databases. If you can’t restore data then your backup is useless. I try to test restoring my backups at least weekly so I know that I can rely on my backups.

Is everyone okay losing data back to the last successful backup?

If disaster strikes you could loose all data between now and the time of your last backup. But all is not lost transactional backups can be scheduled throughout the day, in my case the most data we could loose is 15 minutes. This could be configured to be more or less frequent depending on your data. But remember the previous question and make sure you test a restore of your transactional backups, if you can’t restore from them you will be forced to restore from the last successful backup.

When was the last successful clean corruption test?

Corruption can be a killer if it is not found quickly. If you need to restore to the last backup before corruption occurred this could result in a significant amount of data loss. To check for corruption you need to run DBCC regularly.

Do we have a development or staging environment where I can test my changes first?

If the answer to this is no, then your next job is to setup a development or staging area. Having a development environment makes development a lot easier and I don’t think I could manage to do all the changes I have done recently without one.

Is there any documentation for why the server was configured this way?

I really wish we had more documentation about configurations as it would make finding out why thing were setup the way they are. So unfortunately the answer to this question is No.

What changes am I not allowed to make?

Depending on what your app does, where it is hosted, how quick the changes are needed and many other factors will all restrict what changes you can make. Historical decisions on the database can also affect what changes can be made, if the database has been structured in a certain way, it may be very difficult to restructure it in a more efficient way.

Who can test that my changes fixed the problem?

This is an interesting question, from experience the best people to speak to about problems with the database are the users. If they can show you how to reproduce a problem, you should be able to fix this problem, after that you can probably get them to verify that it has been fixed.

Who can test that the apps still work as designed, and that my changes didn’t have unintended side effects?

This is an extension of the previous question. The main users of your app should be your first port of call to find out if the app works as expected. However exploring side affects and undesired features is something that I would test as part of the development process. It has taken a while but I have constructed a detailed check list that can be used for testing so I know that most bugs can be found before release.

New Horizons Pluto Spacecraft

Tue, 14 Jul 2015 20:00:45 +0000

To explore Strange New Worlds, well today the NASA spacecraft New Horizons has been doing just that. This is the most detailed photograph yet of the furthest planet we have ever explored, Pluto.

New Horizons was launched on 19th January 2006. It has been travelling for 3463 days (9.5 years).

In 2006 we were still using Windows XP and Office 2003. In 2006 I wasn’t even working in IT. As a software developer it is hard to imagine writing software that won’t be used for another 9.5 years.

The spacecraft carries two computer systems: the Command and Data Handling system and the Guidance and Control processor. Each of the two systems is duplicated for redundancy, for a total of four computers. The processor used for its flight computers is the Mongoose-V, a 12 MHz radiation-hardened version of the MIPS R3000 CPU. Multiple clocks and timing routines are implemented in hardware and software to help prevent faults and downtime. To conserve heat and mass, spacecraft and instrument electronics are housed together in IEMs (integrated electronics modules). There are two redundant IEMs. Including other functions such as instrument and radio electronics, each IEM contains 9 boards. The processor distributes operating commands to each subsystem, collects and processes instrument data, and sequences information sent back to Earth. It also runs the advanced “autonomy” algorithms that allow the spacecraft to check the status of each system and, if necessary, correct any problems, switch to backup systems or contact operators on Earth for help.

For data storage, New Horizons carries two low-power solid-state recorders (one backup) that can hold up to 8 gigabytes each. The main processor collects, compresses, reformats, sorts and stores science and housekeeping (telemetry) data on the recorder – similar to a flash memory card for a digital camera – for transmission to Earth through the telecommunications subsystem.

Communication with the spacecraft is via X band. The craft had a communication rate of 38 kbit/s at Jupiter; at Pluto’s distance, a rate of approximately 1000 bit/s. To put this speed into context, the size of the image at the top of this page is 649kb and would take over 86 minutes to travel from Pluto to Earth, (I am sure the actual image was much larger and took even longer to reach us.) Besides the low bandwidth, Pluto’s distance also causes a latency of about 4.5 hours (one-way). The 70 m (230 ft) NASA Deep Space Network (DSN) dishes are used to relay commands once it is beyond Jupiter. It will take 16 months to send the full set of Pluto encounter science data back to Earth.

Most software encounters a problem from time to time, but the software on board this spacecraft has to keep running for almost a decade. Due to the distances involved its not possible to upload any code tweaks. Any commands that are sent to the spacecraft will take 4.5 hours to even get there so there needs to be a huge amount of autonomy and redundancy to cope with such a massive journey.

I can’t even imagine how you would begin to write the software to run such a craft. I don’t think I have written any code that is as old as is running on the New Horizons craft, and if I had I would guess the quality would be quite poor certainly not good enough to work non stop for a decade.

There are some incredibly clever people out there who have been involved in this project. Congratulations team you have every right on being proud as the data starts to trickle back to earth.

How to upgrade to Windows 10

Sat, 11 Jul 2015 20:00:45 +0000

Microsoft are releasing a new operating system on July 29th called Windows 10.

If you have questions about Windows 10 watch this video by Scott Hansleman . He explains how to upgrade from Windows 7 and 8 to Windows 10 and what to expect after you upgrade.

My advice to everyone is that you backup your devices before you upgrade, this is good practice before you install any software or updates.

Microsoft are offering Windows 10 for free for a whole year, so don’t panic. Take your time, backup all your important files and when you are ready install Windows 10. Nothing is going to stop working after July 29th if you don’t have the latest operating system on your machine.

I hope this helps answer some of your questions about what to do at the end of the month. If not feel free to write a comment or contact me with your questions.

Why do we call them bugs?

Tue, 07 Jul 2015 20:00:45 +0000

On my way home from the office I spotted this little chap. In my expert opinion it is a “bug”.

This particular “bug” is harmlessly sitting outside, but have you ever wondered why computer problems are often named after this little chap (and his friends)?

The first recorded bug report was in 1947 when Grace Hopper found a moth between the relays of a Harvard Mark II, this moth was then tapped in the log book as evidence.

Since then the process of removing bugs has been called “debugging”. However bugs had existed in electrical devices before this.

In the early days of telegraphy, there were two types of keyers. The older “manual” keyers which required the operator to code the dots and dashes. And the newer, semi-automatic keyers that would send a string of dots automatically. These semi-automatic keyers were called “bugs”. One of the most common brands of these keyers, the Vibroplex, used a graphic of a beetle.

Luckily the bugs I deal with don’t have legs or wings and are just errors in computer code. When I debug code I need to look at the inner workings of the code, looking for problems, most often it is a typo or piece of logic that doesn’t match up. But the relationship between computer bugs and living bugs does continue, the bug tracking software bugzilla uses a cartoon bug as its logo.

Agile Software Development

Sat, 04 Jul 2015 20:00:45 +0000

I have heard the term Agile Software Development quite a bit, but lets see if we can define it and see if I do any of the processes involved with it.

The dictionary defines ‘Agile’ as mentally quick, or nimble, or quick in movement.

In 2001 The Manifesto for Agile Software Development first introduced the term agile. The manifesto is based on 12 principles.

Customer satisfaction by early and continuous delivery of useful software
Welcome changing requirements, even late in development
Working software is delivered frequently (weeks rather than months)
Close, daily cooperation between business people and developers
Projects are built around motivated individuals, who should be trusted
Face-to-face conversation is the best form of communication (co-location)
Working software is the principal measure of progress
Sustainable development, able to maintain a constant pace
Continuous attention to technical excellence and good design
Simplicity—the art of maximizing the amount of work not done—is essential
Self-organizing teams
Regular adaptation to changing circumstance

Lets go through this list one by one and examine what I understand of it and how I might have used it in the past.

Customer satisfaction by early and continuous delivery of useful software

My boss bombards me with hundreds of changes that she wants straight away, I keep her reasonably happy by giving her regular small improvements. Yesterday I released some changes, and two weeks before that I released another chunk, on Monday I will find out that I haven’t included feature Y, so in another couple of weeks feature Y will be delivered. This is my understanding of delivering continuous improvements to my software.

Welcome changing requirements, even late in development

Lets ignore the word welcome, as changing requirements are really annoying. I designed a system to track how far through a 5 step process you were, after I deployed that change it was discovered that the process was actually 18 steps. Annoying as this was, my design allowed the easy addition of these extra stages.

Working software is delivered frequently

Yes, I do this, see my comments above. If you make a single change and then deploy it, bug fixes are easy as most of it is still in your head. If you spend 6 months making 1000 changes, it will take you extra time to remind yourself what you did and why.

Close, daily cooperation between business people and developers

I don’t do this daily but I regularly demo my changes to other departments and directors so I can gather feedback and further revise my changes. Before I start work I always try to understand what is needed and how they want it to work and try and figure out what they need not what they want (different things)

Projects are built around motivated individuals, who should be trusted

Not sure I understand this one. There are people that use my projects that I always go to, to find out how it is being used. There are also people that are not interested in change, it is always far more difficult to convince them of the value of what I am working on.

Face-to-face conversation is the best form of communication

When I was an IT support guy, I used to like going up to someone’s desk and seeing them experience the problem, it could usually be fixed in a couple of clicks. However when you had a call from a remote office and had to talk through trying different things it could take hours to fix. The same can be said of deploying a new feature, if you talk to your users and explain what has been added they are far more likely to use it, than if you send them an email listing what you have done.

Working software is the principal measure of progress

If a Manager can click two buttons to find out how busy their department is, when before they only had out of date spreadsheets, this is a clear measure of progress. Any time spent on code that never gets deployed doesn’t improve anything, its only when that code starts being used has progress been made.

Sustainable development, able to maintain a constant pace

I am pretty sure I don’t do this. I work harder at some times than others. Sometimes there are other projects or other demands on my time and so development on a particular project will decrease and there are times when I eat, sleep, dream about a project.

Continuous attention to technical excellence and good design

I do strive towards good design and with each deployment aim to improve the quality of code in production. However my limited knowledge means that I don’t always produce the best code but it is improving. If I look at some work I have done in the past, I would definitely do some of that differently if working on it now.

Simplicity—the art of maximizing the amount of work not done—is essential

When I first read this one, it made no sense to me. However I believe it can be restated as minimising the amount of work done. Recently I was writing SQL code and I had about 4 or 5 different queries to produce the report that was needed. Every time the requirements changed it was a long slog through each of these queries to update them. In the end I had to rewrite this whole process using one query, it took some time but was simpler and is now easier and quicker to maintain. The simpler the code is the easier and quicker it is to maintain, and you have less work to do.

Self-organizing teams

Not sure this applies to me as I work in a team of one. I do try and keep myself organised but not sure if that qualifies as self-organizing.

Regular adaptation to changing circumstance

Very similar to the second point about changing requirements but I think also includes changes required due to problems you encounter along the way. A simple example of this is when you encounter a problem, there are lots of different ways to solve it. If you can’t solve something in the database layer, then it might be solved with a script or in the user interface.

So in summary I am not only a developer but very close to being an Agile developer as well, assuming these 12 points are what is needed to be ‘Agile’

Developer Personality Test

Thu, 02 Jul 2015 20:00:45 +0000

I was sent a link to a developer personality test earlier. The test aims to give you a look at your developer personality similar to the famous Myers-Briggs personality test. A shorter version of this test can be found here .

Myers-Briggs tells you if you are Introvert (I) or Extrovert (E), Feeling (F) or Thinking (T), Perception (P) or Judging (J) and Sensing (S) or Intuition (N).

This test however tells you if you are:

Independent vs Collaborative

Abstract vs Low-Level

Frontier vs Conservative

Generalist vs Specialist

Idealist vs Pragmatist

Independent: You prefer to spend most of your time working in an isolated environment. You rarely want to collaborate because you have a better chance of solving problems on your own. If you do have to collaborate on the direction of a project, you dislike it when you have to defend your position or when others try to muddy your plans for the project. It’s better to have one strong vision for a project from the lead programmer on that project. Having a large team or allowing others to have significant control over the project risks communication errors, a muddied code base, and inefficiencies. If one developer has a firm grasp on the entire codebase, they’ll be much better at fixing and improving it quickly.

Collaborative: Good code doesn’t happen in a vacuum. You need a team to keep you energized and on your toes so that you’re constantly improving the project code using the entire team’s varied experience and strengths. You like to talk frequently with colleagues and discuss ideas so that you can learn from others and spark ideas. It doesn’t matter what their experience level is, you can always learn something by including everyone. A solo coder can’t possibly code a large software project with the speed and quality that a team can.

Abstract: You prefer to write in languages and frameworks that simplify development as much as possible and use as few characters and lines of code as possible. The trajectory of software development has always been toward making life easier and error-free. Programming has always been about adding more abstraction layers to simplify tasks, so why not trust the technology on your layer and don’t worry about handling the lower layers?

Low-Level: The more abstraction tools and high-level languages we build, the further we get from understanding and controlling the deeper components in our programs. This means lower performance and endless bug searches. Developers today need to have a stronger understanding of compilers, memory management, pointer arithmetic, and other low-level topics.

Frontiers: You like to work at the cutting edge. Using too many old languages and technologies bores you, and it severely hinders your software’s potential to keep outdated technologies in it for too long. Developers need keep their ears to the ground for new technologies and new versions of tools that they already use. Even if the community and maturity of the project isn’t at a level that most enterprises would consider “safe,” you’re willing to be an early adopter if you believe the community and the technology has momentum. Development technology is changing faster every day, and we need to constantly be adopting new tools and methods to keep up.

Conservative: It seems like every 10 years we forget all the problems we solved in the previous decade and start to build ‘new’ tools that solve the same problems, even though there is a perfectly good solution that has existed for years. Enterprises have it right when they make conservative decisions about their technology stack. Why would you hang your business on a technology with only a few years of maturity in just a handful of production use cases? Technologies like PHP, Java, and SQL have been mainstays of the development industry for years, and it takes a lot of time for new technologies to make it into that maturity tier.

Generalist: You like to be known as a “Jack of all Trades” and a reference for others on your team. You jump at every chance to enhance your skills in a wide variety technology topics, not just the ones that apply to your day to day work. You don’t always know when it might be useful to have these extra skills, but when the time comes, you are ready. If more developers took the time to learn other skills outside of the ones relevant to their project, they’d work more seamlessly with the rest of their organization because they’d have more empathy and understanding of the challenges that their colleagues face.

Specialist: If you’re a Jack of all Trades, you’re a master of none. Mastery in one or two areas is what makes you valuable. What’s the point of learning skills for other jobs you don’t have and can’t control? When you plan for a project or do technical research, it’s always focused on something you’re working on. You don’t get side-tracked. If you learn a new skill, it’s because the project requires that you do it. Most or all of your hobby projects are also building your mastery of the skills you use at your job.

Idealist: You believe in the power of well-defined processes. It’s crucial to an organization’s success that they create and follow appropriate and effective processes for building software. Trying to improvise or play it by ear invites the possibility of workflow errors that can decrease the quality of the software or even cause major product failures. Planning is also extremely important to you. You like to research all of the things you will need to know before starting a project. It’s important to find out the best architecture for your software beforehand, and then strictly implement that architecture with objective best practices. The more planning and scaffolding you do in the beginning, the less overall work you will have to do to complete the software.

Pragmatist: Speed is your best weapon in a competitive industry, and to quickly prototype and build new products, you need to have a flexible, pragmatic process. You don’t have a few months to plan your projects, you need to just start coding and a good path for the project will reveal itself. Great products are made through frequent feedback and releases, so why shouldn’t your plan be just as adaptable? Your plan should be adapted to changes in the software, and your expertise should be adapted to the project. You shouldn’t spend your precious time studying a problem that you’re not certain to run into while coding your projects. Trying to build test coverage for every possible scenario and having long meetings throughout the process are a waste of time and distract you from doing more productive work.

I am not going to admit where I came in this test as being a newbie developer my answers are very subjective and I don’t believe I have enough experience to really say where I lie on many of these issues.

I only have experience of working on my own and in very small teams, while I think working in a bigger team would have benefits I have no experience of this.

Sudoku Challenge

Fri, 26 Jun 2015 20:00:45 +0000

Last night I went to a Code Dojo at Leeds Sharp (the coding user group I have started going to). A Code Dojo is a programming challenge that people work on usually in pairs.

The challenge that we worked on was to code a solution to the puzzle game Sudoku. The code we worked with can be found on github.

I don’t know much about Sudoku but the game goes like this. You start with a 9×9 grid, and the idea is to fill in all the missing numbers.

Each Row can only use each digit 1 to 9 once, each column can only use each digit 1 to 9 once and each subgrid (3×3) can only use each digit once.

I worked with Richard one of the Leeds Sharp organisers, we started by trying to loop through each empty cell and insert values in a brute force attack, but this approach didn’t get us very far (literally as the application crashed!)

The approach that we needed to use was to loop through calculating possible values.

It was only when I sat down at this dojo did I realise how much I still have to learn, but while talking with Richard about different approaches and different coding structures I started to learn.

Fathers Day

Sun, 21 Jun 2015 20:00:45 +0000

In the UK today is Fathers Day. Happy Fathers Day Dad!

I thought I might talk about how he encouraged my interest in technology, especially as in about 8 weeks I will become a parent and will have someone to encourage and inspire.

I am going to sound really old but when I was born there was no internet, no mobile phones and computers more often appeared in sci-fi than in your home.

The first computer I remember us owning was a Lynx. A quick google reveals there were a number of different models, I have no idea what we had but here is what I found.

In March 1983 the Lynx 48K was released, in September this was upgraded to 96K. In December 1983 a new Lynx was released with 128K. To put these numbers in perspective on my desktop is an image file that is 123K! The CPU in this computer was rated at between 4 and 6Mhz. (Modern PCs have CPUs around the 1.5Ghz or 1500Mhz)

The Lynx we had, didn’t have a disk drive or a mouse. Programs were loaded from audio tapes (Audio tape was a form of data storage that came before CDs where data was stored on magnetic tape, most often used for music) The tapes we had would often get stuck and data wouldn’t load. The answer to this was to fast-forward/rewind the tape or hit the tape on your leg. For some reason this is a memory I still have of my dad hitting a tape on his leg to get a game or something else to load on the lynx. We also had a joystick for playing some of the games. I believe we also had an early thermal printer.

My memory of the Lynx is in black and white. The reason for this is that the Lynx was connected to an old black and white TV, it was a special treat when we connected the main colour TV to the Lynx.

We had a wide range of games for the Lynx, I can remember Jet Set Willy, and Diggerman. But there were also programs which my father had created. The Lynx had a form of BASIC where you can write you own programs and my father spent many hours writing programs for me and my sister. (We actually never realized this at the time)

With that as my introduction to computers is it any wonder that I do what I do now? Now what should I do for my child to encourage them in a similar way?

As a family we never owned a games console. Not sure if this was a financial or moral choice but it is something I will have to think about. I want my child to be knowledgeable about computers and able to compete in the competitive job market, but I also want them to know about science and nature and to enjoy their childhood. Lots of technology choices to make over the next few years.

50th Blog Post

Wed, 17 Jun 2015 20:00:45 +0000

WOW! I have reached the milestone of 50 blog posts.

I have tried to blog at least every week and I have learnt a lot. I have talked about my work, technology, things I have learnt, things I want to do and many more things. It is too soon to tell if my blog is going to be a success, but I think I am starting to find my ‘blog’ voice and I intend to keep on blogging.

To celebrate here are some of my highlights.

User Groups and F# By far my most popular blog post. I talk about my first visit to the Leeds Sharp group and an introduction to the F# language.
Monitoring Screens I talk about creating a big screen of vital monitoring information.
Ten Forward Episode #135 – Simon’s Desert Island Trek My appearance on the Ten Forward Podcast
Building something with a Raspberry Pi I blog about getting a Raspberry Pi to play around with
Improving your Blog John Sonmez tries to help my blog improve
Javascript progress About learning javascript
The Raspberry Pi Adventure Starts I get a Raspberry Pi and start building something with it
Weakest Database Design I talk about database design
Trying Windows 10 I talk about my experience with the latest windows operating system
Laziness I talk about the extreme ways IT Professionals go to be more lazy

Requirement Gathering

Tue, 16 Jun 2015 20:00:45 +0000

Why is it so hard to find out exactly what is needed when designing changes to a system?

When adding functionality to databases I always like to dive in and start adding extra columns and new tables. But at some point you need to find out what is needed.

The process of gathering requirements that I have followed goes something like this.

My boss tells me that we need a system to solve x problem. This is the broad overview of what is needed.

Next I need to find out specifically how the system is being used and what things need adding. Discussing the specifics with my boss often results in a high level of confusion. I always try to speak with the users that will be using this system on a daily basis. If I create something that isn’t liked people will avoid using it and it won’t solve the original problem so I always think it is very important to speak to the day to day user.

After I have an idea of what is needed, I build something. Once I have something that sort of works, I will try and demonstrate this to the user to gather feedback. This feedback is invaluable as it will often reveal if I am going in the right direction and reveal missing requirements that need to be incorporated in the finished solution.

I will often repeat the last stage a few times especially if there is a big change needed, once I am happy I will demonstrate to my boss before deploying the solution.

After deployment there is often a period of fixing issues and gathering feedback before I can consider the project finished.

This whole process is very time consuming as there is a period of changes being tweaked back and forth following feedback. A better way is to gather a detailed specification of what is needed and work towards delivering that.

Runaway SQL Log growth

Fri, 12 Jun 2015 20:00:45 +0000

Today is my day off, but I wake up and have a quick look at nagios to see if there is anything I need to worry about. Yes there is, SQL Server has run out of disk space on its data disk.

I race downstairs and VPN onto the server to find out what has happened. One of my monitoring databases has had runaway log growth and is over 80Gb is size.

BACKUP LOG [DBName] WITH TRUNCATE_ONLY  
DBCC SHRINKFILE('DBName_Log')

Free disk space is back to normal, all users will be unaware of the problem and everything is fine again. I create a daily job that runs the above code, that way it should stay a manageable size.

Next I need to find out why it happened and to prevent it happening again in the future (Next time I have a day off I want to lie in!)

I check the SQL logs and notice

BACKUP LOG WITH TRUNCATE_ONLY or WITH NO_LOG is deprecated. The simple recovery model should be used to automatically truncate the transaction log.

Then I remember what I have done to cause this issue. I have a separate disk for my backup files and earlier in the week I noticed this disk was filling up, a large amount of space was taken up by transactional backup files. I thought I don’t need to backup the transactions for this non critical database, I will just do a full backup at the start of everyday.

However what I forgot is that a transactional backup keeps the log file under control, once this backup was stopped the log file grew uncontrollably. The answer, change the database from FULL mode to SIMPLE.

This is my understanding of how backups work in FULL mode. A full backup is done at the start of the day which resets the log file, then changes in the database are stored in the log file, this is backed up into a transactional backup and the log file gets reset. If you have regular transactional backups throughout the day the log file doesn’t grow too much, however with no transactional backups your log file contains an entire days worth of changes and so for a monitoring database this could be quite large.

In SIMPLE mode you can’t do transactional backups and the log doesn’t grow uncontrollably. This shouldn’t be used for production databases as if there is a problem you could loose data.

Don’t Live with Broken Windows

Mon, 08 Jun 2015 20:00:45 +0000

Last night I started reading The Pragmatic Programmer by Andrew Hunt and David Thomas .

One practical tip that immediately jumped out at me as useful in my quest to be a better programmer: Don’t live with broken windows.

The theory goes like this: an abandoned building with a single broken window attracts additional broken windows, graffiti, litter, and a sense that no one cares about this building. However this can be avoided if the authorities repair the window straight away.

You can apply this theory to other walks of life, too, like keeping your kitchen clean. After you’ve spent hours cleaning your kitchen to make it spotless, you make an effort to keep it clean. But if your kitchen is piled high with dirty pans, leaving out an extra mug or dirty bowl isn’t going to feel like it makes any difference.

The software engineering implications of this theory are that if you take care to keep your code tidy, it will stay that way. If you know something is broken but never make time to fix it, other bugs and issues will creep into it, until it’s a mess that no one dares touch.

We have a website that is starting to get a bit buggy. It’s not broken, but it’s been a while since anyone has given it any love. Bits of its functionality are starting to get flaky.

Being the only programmer in my place of work, I seldom have time to think about this website — let alone sit down and maintain it.

But the number of related support calls have been gradually increasing — more windows are breaking — and I’m spending more and more time patching its bugs to keep people happy.

Know that I’m aware of the broken windows principle, I know exactly what to do: sit down with this website for a couple of days and find out what exactly is causing these issues.

Then I’ll rebuild the flaky bits, replace the proverbial cracked glass, and wash down a few walls while I’m at it.

I’m a developer now

Wed, 03 Jun 2015 20:00:45 +0000

For a while now I have been at the door of describing myself as a developer, but I think I have finally gone through that door.

It is all a state of mind, its not about having x amount of knowledge, its just about saying “Yeah, I’m a developer now.”

One thing that pushed me through that door, was an application form for something in which I ticked the software developer box instead of the IT Manager box.

The other thing is some of the tasks I worked on recently. One of our databases has a Web Service running that supplies a website with some figures, and we found out recently that this needed changing due to a change on this website. Our contractor quoted us to do this work, I think it was 8 hours. My boss said why don’t you have a go at doing it instead and save us money.

So I had a go, thinking this will take me a couple of days at least, as I will have to figure out how the code works and end up not understanding it. But no I had the changes done in less time than the contractor had quoted us. Go Me! I will say that these changes have yet to be fully tested, so I still might end up spending more time on this.

User Groups and F#

Sat, 30 May 2015 20:00:45 +0000

For a while I have been threatening to try a user group. Well this week I went to my first meeting.

The user group I went to is called Leeds Sharp who are celebrating their third birthday, happy birthday guys and thanks for the cake.

The talk was all about F#. I know nothing about F# other than it is in visual studio. The speaker was Phil Trelford an expert in F# and former game developer who regularly speaks at conferences. Phil is an expert programmer, one of his stories was that while he had a broken arm, he wrote a copy of Excel in F#. Even with two hands I wouldn’t know where to start on something like that.

There are plenty of advantages to using F# over C#, fewer lines of code was one. In Phil’s example he had some C# code which was about 30 lines but the F# was 2, part of the reason for this is F# has no {}.

I found myself thinking this must be too good to be true there must be some disadvantages to using F#?

I don’t know what they are yet, I am guessing you need to play around with the language a bit to really understand them.

One claim that was made was that F# has less bugs. I am unsure how this can be true. Code is written by humans, humans will make mistakes which ever language they choose, there will be bugs.

After the group I came away with a few things to try out. Try the F# tutorial that can be found in Visual Studio, try a Koan, this is a piece of code that contains failing tests, in the process of fixing the tests you learn the language, come back next time to Leeds Sharp and learn something else.

Visual Studio

Tue, 26 May 2015 20:00:45 +0000

I recently replaced my installation of Visual Studio 2013 with Visual Studio 2015 RC.

I like the new version, I am not a Visual Studio expert so it will probably take me a while to find all the good stuff but here are some initial thoughts.

Being as my MSDN subscription is still valid I have installed the professional version to take advantage of its extra features like CodeLens.

One of the first things I spotted was that the integration with Azure has been improved. In the last version it was difficult to sign in to Azure with the correct credentials, especially if you have more than one Azure account. Now you can add multiple Accounts and Subscriptions.

CodeLens is a feature that has been around in Visual Studio since 2013, but in 2015 it is available in Professional meaning more people have access to it.

CodeLens gives coders useful information at a glance. Above each class/method is listed how many references there are. If you click on the number of references you can see where that class or method is referenced in the rest of your code. Useful to be able to see which methods or classes are not being used.

Next CodeLens shows who (according to git) last changed the class or method and how many days ago that was. Clicking on it shows a cool graph of when changes have happened and who did them.

Next you can see the number of changes that have been made, basically a source control history, but without having to load up your git client.

For code that doesn’t contain classes or methods such as T_SQL you can see at the bottom of your code window the last two CodeLens information to help you track down what changes have happened recently.

The last new feature that I have noticed is the Light Bulbs that keep showing up all over my code. I think the Light Bulbs might be called Quick Actions, but whatever they are called they are suggestions on how to improve your code. So far they have suggested to be to get rid of using references that are not used, simplify a fully qualified name, drop unneeded this keywords. I am sure more will popup as I do more coding.

These improvements to Visual Studio I like, and I am sure there are many more that I haven’t noticed. I expect support for the vNext .net framework is also in there somewhere which hopefully I can play around with soon.

Overflow

Fri, 22 May 2015 20:00:45 +0000

Today I encountered a new error.

Run-time error ‘6’: Overflow doesn’t really tell me much. The error was occurring in the Access ADP front-end of our main database. It was only occurring for one particular Id number which was really confusing me.

First thing I tried was to remove the most recent changes I had made, this made no difference. So time to look at the VBA code that was run just before the error.

Dim iPropertyId As Integer
iPropertyId = DLookup(“PropertyId”, “Survey”, “[ProjectNo]=” & Me.ProjectNo)

All looks good, even running the Dlookup query in SQL Management Studio threw no errors.

SELECT PropertyId FROM dbo.Survey WHERE ProjectNo =

Next thing I tried was explicitly setting iPropertyId to the problematic Id, still the same error.

This made me look at the error message again. Overflow suggest something being too big, but what could it be.

Integer has a size limit of 32,767 and the value I was trying to pass into iProperty was 32779. This was what was causing the problem, my database had grown to such a size that Integer was too small, I needed to change it to a Long Integer. Long has a size of 2,147,483,647 so my database should keep working for a while more.

The famous programming help website Stack Overflow is named after such an error. This error shows how difficult it is to decide what variable type to use, as you don’t know how quickly your database will grow.

Trying windows 10

Sun, 17 May 2015 20:00:45 +0000

My laptop has been getting a bit cluttered up with stuff so this weekend I gave it a reset.

As I am using windows 8 this was much easier than doing a format reinstall which I have done many times with previous OSes. Open settings and select recovery and after about 4 hours everything was back to factory settings.

I then installed a few updates and then installed the windows 10 preview. One of the requirements for windows 10 is Windows 8 Update (the one that came after 8.1), it took a few reboots and updates until I got that installed.

I had purchased the pro pack which gives you the same features as windows 8 pro but I didn’t notice until after I installed 10 that this hadn’t been installed. I will need to roll back to windows 8 apply the update and then reinstall 10 again. I can’t really be bothered to do that, so I will live without HyperV for now. As it is I have done a lot of sitting around waiting for things to finish installing.

I much prefer Windows 10 to Windows 8, an earlier preview had an issue with drivers for my second screen but that appears to have been resolved now. The start menu is back and I prefer it, to being taken to that extra screen of options that windows 8 had.

I prefer the email client that windows 8 had to the windows 10 version as my work emails don’t work with it. I assume this is due to how my exchange server has been configured and maybe a later preview will allow my email to work or failing that I will have to get emails at work fixed properly.

All apps now open in windows which is much nicer. Cortana is very similar to Google Home on my phone where I can talk and ask it stuff. But I never use this sort of feature apart from testing it.

The new web browser has some nice features like the option to scribble over a web page but nothing that special that I think I will change my default browser from Chrome. It will be interesting to see how it performs in browser stats.

I am part way through installing Visual Studio 2015 RC, but I think that will have to be a separate post.

As A Service

Sat, 16 May 2015 20:00:45 +0000

In Cloud Computing there are a lot of terms that end aas or As a Service. Most of these I hadn’t heard of until I started writing this list.

Any service that is delivered over the internet instead of hosted locally on your network or PC could in theory be described as an As a Service.

PaaS Platform As a Service

This is one of the big ones. Microsoft Azure provides a Platform as a Service which I am familiar with. Platform as a Service is where a provider provides a platform where you can build apps or websites.

SaaS Software As a Service

Another popular one. Software as a Service can be as simple as a website that runs a service that a customer wants to use, I work for a company that provides a SaaS product.

IaaS Infrastructure As a Service

The last of the big ones. A good example of IaaS is a Virtual Machine which can either be hosted on a server somewhere (a private cloud) or on the internet via a company such as Azure (public cloud) The Pizza as a Service diagram illustrates the differences between Saas, PaaS and Iaas.

NaaS Network As a Service

This is just a type of IaaS that specializes in providing networking. Anything that provides network connectivity could be included in this category.

CaaS Communications As a Service

Another subtype of IaaS this time specializing in communications, this could include Voice over IP or other similar technologies.

MONaaS Monitoring As a Service

If you have a SaaS, PaaS or IaaS you will most likely want to monitor that it is working, I certainly do. This is something that is often included in your IaaS or PaaS package. Azure has various tools for monitoring and this could be included in this category.

BaaS Backup As a Service

With the growth of cloud storage and the decrease in its cost, backing up to the cloud is a very attractive option. Any service that allows you to backup and restore from the internet can be included in this category. Your provider needs to manage your backups for it to be truly BaaS rather than just another place to store your files.

DaaS Desktop As a Service

This is where your desktop is visualized and stored in the cloud. I know very little about this as I have never used it, but I would imagine you need a strong internet connection for it to work reliably.

DBaaS Database As a Service

This is a simple one if your database is stored in the cloud like Azure SQL Database then it fits into this category. If you run your own sql server install on a VM then it doesn’t fit in this category as you are still managing it yourself and is IaaS

HaaS Hardware As a Service

HaaS this is another subcategory of IaaS which concentrates on hardware.

IDaaS Identity As a Service

This is where the management of who you are is managed in the cloud. Single Sign-On could be achieved if a website redirected determining if you are who you say you are to a particular IDaaS. Azure Active Directory is an example of this.

SaaS Storage As a Service

You get the idea now, storing files on a remote cloud product is an example of Storage as a Service. DropBox or OneDrive are good examples of this.

FoaaS F Off As a Service

.NetRocks mentioned this a few weeks ago and is a joke As a Service. http://foaas.herokuapp.com/ The idea is that you can use this service to tell people to F off.

Writing this blog post has given me a better understanding of all the aaS that are out there. I am sure I haven’t explained some of these very well and no doubt missed some off.

Top 25 Star Trek Characters

Sun, 10 May 2015 20:00:45 +0000

I am going to take a break this week from talking about technology to talk about Star Trek.

I have just listened to a podcast where they consider this list of great Star Trek Characters. I am now going to try and come up with my own top 25 characters. Here we go.

Spock. My number one star trek character is Spock. Leonard Nimoy who recently passed away brilliantly brought this character to life. We have the inner turmoil that many of us can associate with, we have a journey to become more human, we have the pursuit of logic. We have the great interplay between him and McCoy and Kirk.

Data. My second number one star trek character, well OK, second star trek character is Data. His weekly exploration of what it meant to be human is star trek’s best exploration of the human condition. There are so many moments which make me like Data but I think when I builds his daughter, Lal is possibly my favourite.
Picard. The Captain of the Enterprise. You want to know about leadership or diplomacy you watch Captain Picard in action. If only more of our world leaders could be like Captain Picard we would have all our Earthly problems sorted by tea-time.
Kirk. The original captain of the Enterprise, not so much a Diplomat but could talk a computer to death in minutes. A fun guy to be around as long as you didn’t work in security or want a long lasting relationship with him, (sorry ladies this guy is married to the Enterprise)
Worf. Our favourite Klingon. The most honourable man on this list and could handle himself in a battle. Also popular with the ladies if Troi and Dax have anything to say about it.
Sisko. Our third captain, but Sisko had much more on his plate than the others. He had a Son to bring up, he lost his wife, he was in the middle of the biggest war the Federation has ever seen (at least on screen anyway), he became a religous icon and later a God.
7 of 9. The sexy borg from Voyager what’s not to like? Also brought some much needed conflict to the Voyager crew.
Garak. The plain and simple tailor from DS9. But is he also a spy? There are so many layers to Garak you have no idea what to believe.
McCoy. Bones to his friends. A great Doctor but even greater friend to Jim Kirk and Spock (not that either of them would admit it.)
EMH. Please state the nature of the medical emergency, started the show off as just another piece of technology and ended up one of the crews most important characters.
Janeway Coffee Addict and defeated the Borg our first female captain.
Dukat. DS9s regular bad guy. Started off Bad, then softened to be a good guy for a bit and then signed an alliance that started a war, in the middle of that he went crazy and ended up going to the Bajoran Hell
Weyoun. The leader of the dominion from the Gamma Quadrant. Was a clone so he could be killed off a few times and still come back. Had some great interplay with Dukat and later Damar.
Damar. Started life as Dukat’s lieutenant and rose in rank to leader of his people, before switching sides to lead a rebellion.
Scotty. You cannae change the laws of Physics. An engineer that weekly performed miracles, more often because or large repair estimates.
Barclay. The only Starfleet office that was unsure of himself and couldn’t talk to people (Oh a bit like me then)
Riker The Captain Kirk of Picard’s crew, could inspire his crew and was popular with the ladies, but looks better with a beard.
O’Brien. The everyman of DS9. He constantly had a station that was falling apart but he managed to keep it going, a bit like how I feel sometimes.
Quark/Rom/Nog. These three Ferengi made the Ferengi good again. Nog started off as comic relief and ended up a Starfleet officer after turning his back on the acquisition of profit. Rom like his son turned his back on profit to fix things, and Quark was the businessman always had his eye for profit.
Odo. The shape shifting outsider that maintained order. Was truly an outsider with no idea where he had come from, but eventually he came to tolerate and even love us Solids.
Martok Klingon with one eye that welcomed Worf into his house after his second dis-commendation.
Zefram Cochrane Without this guy there would be no Star Trek. Invented the warp drive and enjoying to drink.
Uhura Just a communications officer, but in reality was so much more. One of the first black women on television and inspired a generation to better themselves, was the other half of the first inter-racial kiss
Mirror Universe Hoshi. Our universe Hoshi Sato was timid and scared of warp drive, the mirror universe version ended up empress of an entire Empire.
Kai Winn Power currupts and Absolute power corrupts absolutely and Kai Winn is a prime example of this, so blinded by power she couldn’t hear what her precious Gods were telling her and ended up sleeping with he arch enemy.

Using your Rubber Duck

Mon, 04 May 2015 20:00:45 +0000

Rubber Duck is a process in development where you talk through an issue with an inanimate object such as a Rubber Duck. It gets its name from the book The Pragmatic Programmer where it was first used.

I am the only programmer where I work and often I find myself with a question that I can’t find the answer to. I do however have a few friends who are far more knowledgeable than I am about this sort of thing. I often find myself explaining my problem via an email to them. During the process of writing this email I have to explain my problem and this sometime reveals the problem and I don’t even need to send the email. At other times by the time they have replied to me, my brain has moved on with the problem sufficiently that I am close to a solution if I am not already there.

Other people use different items as their ‘Rubber Duck’, people on the code newbie forum have used items such as a cat, stuffed octopus, a Lego minfig, or just plain old talking to yourself.

Being the only programmer in my office is difficult. A simple what do you think of this question that could be asked across the office has to be done inside my head and I may think a needlessly complex solution is better than a simple one just because I haven’t thought of the simpler version yet. There are of course advantages to being the only programmer, I can do what I want and learn any technology I like, but I am starting to think the advantages of programming in a group are outweighing programming alone.

I might actually like SQL Server

Thu, 23 Apr 2015 20:00:45 +0000

How did this happen I think I may actually like SQL Server now?

I remember a few years back when I used to grumble about writing a SQL query that included a JOIN. For some reason back then I could never get my head around JOINs.

Today is a totally different story. I wrote a deployment script that did the following:

dropped a database,
restored a database from a backup file,
Created a database table,
copied the contents of a table into this table,
disabled and re-enabled some triggers,
dropped some constraints and columns from a table
and renamed some columns.

It’s not like I woke up this morning with SQL Server knowledge. For many years I have been adding extra features and changing functionality to our line of business database and over time my SQL confidence has grown.

For every single one of the steps above I googled and looked up the SQL syntax (every time I write an insert or update statement I look up the syntax, one day it will stick in my brain!) I think the main reason is once you have used SQL for a while you get to see how it works and can split it down into small steps.

As I blogged the other day I am currently working on improving a bad database and today I wanted to test the deployment process. All my changes are in a SSDT project so I took a backup of my database and tried to publish.

Error! Your changes will result in data loss, no surprises there as I was expecting that error. The main culprit for this was a trigger I wrote but I didn’t find that out until the end of the day.

As I was working on a backup I could do what I like including destructive changes. I tried creating a pre deployment script. I thought if that runs first, I can persuade SSDT that it matches SQL Server and therefore no data is being lost.

This didn’t work, but I had created the start of my script mentioned above. I had steps 3 and 4. Lets try running my script first and then try running the deployment of my SSDT project. To get this to work I am going to have to rebuild my database a few times, I did this manually with SQL Management studio a couple of times until I thought, I could add this to my script – that’s step 1 and 2 done.

Then I got some errors about triggers so I tried disabling them and re-enabling them after it had finished. That’s step 5.

Finally I needed to drop some columns, this resulted in an error about a constraint. A few trial and error run through’s to find out which constraints I needed to drop and then I could drop the columns.

The last step was to rename some columns. I thought after this I can run publish from visual studio. No it still complained about data loss. I dropped another trigger and then I could run everything without any errors.

Wohoo! Aren’t I clever. A simplified version of my code is below

USE master -–can’t drop the database if its open!
–-Step 1
DROP DATABASE DBName

–-Step 2
RESTORE DATABASE DBName FROM DISK=’E:\DBName.bak’
WITH
MOVE ‘DBName_dat’ TO ‘C:\Program Files (x86)\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\DATA\DBName.mdf’,
MOVE ‘DBName_log’ TO ‘C:\Program Files (x86)\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\DATA\DBName.ldf’,
REPLACE,
STATS=10
USE DBName –-swap back to the database you just restored

–-Step 3
CREATE TABLE TName2 (
ID INT NOT NULL,
Name NVARCHAR(50) NOT NULL,
Address NVARCHAR(50) NOT NULL,
City NVARCHAR(50) NOT NULL,
Postcode NVARCHAR(50) NOT NULL)

-–Step 5
DISABLE TRIGGER TR_Trigger ON TName1
GO

–-Step 4
INSERT INTO TName2 (Id,Name)
SELECT Id, Name FROM TName1
GO

–-Step 6
ALTER TABLE TName1
DROP CONSTRAINT [DF_Id], [DF_Name]
GO
ALTER TABLE TName1
DROP COLUMN Id, Name
GO

–-Step 7
sp_RENAME ‘TName1.City’, ‘Area’ , ‘COLUMN’
GO

–-drop that last trigger
DROP TRIGGER TR_Trigger2
GO

–-Step 5
ENABLE TRIGGER TR_Trigger ON TName1

Weakest Database Design

Tue, 21 Apr 2015 20:00:45 +0000

I have been recently working on a database that hasn’t been designed but has been hacked together by lots of people over the years.

It started life as an Microsoft Access database created by my director to keep track of all the projects that our company does. It was not much more than one table listing everything. At some point the data and frontend was split with the data being stored in SQL Server and the front end being an Microsoft Access ADP.

Over the years more and more columns have been added to this table that it long ago became pretty much unmanageable. A few normalized tables exist but for the most part everything is stored in this one table which makes querying it very difficult. At the last count there were 81 columns in this one database table.

Attempts to improve this database have been tried before but due to the fact that the whole company basically runs off this database and no one person knows everything that the database needs to do causes any changes to be very difficult. I suspect it may be beyond my abilities but I am going to give it a go.

One advantage I have is version control. All the tables and the structure is in version control, so when I break things I can roll them back. Any changes to the front end are also in source control, these are harder to roll back in case of problems but not impossible.

I have met with department heads to try and establish which of the 81 columns are being used and which could be dropped. Database design shouldn’t be a democracy, your databases should be designed in a way to store your data most efficiently. In this case it felt a bit like the database was playing “the weakest link” with everyone voting off unpopular columns. “You are the weakest database column. Goodbye!” After much discussion I have over 30 columns that can be dropped and 10 or so that require further investigation.

Using the Visual Studio project of the database, I removed the columns. This generated thousands of errors as most columns are included in loads of views and stored procedures. All these references need removing, so not a small job. This is made worse by the fact that the views and stored procedures are in a mess, and it is very hard to read the T-SQL code and see what is doing what. Part of my next job is going to be tidying up all these queries so I can see what’s what.

The advantage of using the Visual Studio project to do these changes is that I can generate a SQL script of the changes and I can rerun a few times and test restoring different columns as it is unlikely to be a smooth deployment.

Security Camera with Raspberry Pi Camera

Wed, 15 Apr 2015 20:00:45 +0000

So my first Raspberry Pi project is a security camera. If you have read my previous Raspberry Pi posts you will know that I have been playing about with the camera but I had one or two problems with Pidora so I am using Raspbian for this example.

To take a photo with the camera module you can use the command raspistill, this has lots of options you can view the full list by just running the command with no arguments. I will list a few options I have used.

-o filename : this outputs an image to a file with name filename -vf : vertical flip -hf: horizontal flip these are useful if the camera is upside down etc -q number: quality of jpg image between 0 and 100, the default image size is around 3 Mb, I reduced it to quality = 10 which gave me an image size of 300Kb which is more reasonable -a text: annotate you image with some text, I found that the text needed to start with a character, numbers only failed for me.

I want to script the creation of images, so I decided to name the files my script created with the time and date.

This is fairly easy with a bash script.

DATE=$(date +”%Y-%m-%d_%H%M”)
raspistill -q 10 -vf -hf -a a$DATE -o /var/www/$DATE.jpg

The first $DATE is the date I have added to the image itself, because of the problem I encountered with annotating numbers only I have added an extra a to the start of this.

It is now very easy to get CRON to run this script every few minutes, I decided to run it every 2 minutes.

My next script tidies up so you don’t get overrun with images.

cd /var/www
ls *.jpg > stills.txt
mencoder -nosound -ovc lavc -lavcopts vcodec=mpeg4:aspect=16/9:vbitrate=8000000 -vf scale=1920:1080 -o timelapse.avi -mf type=jpeg:fps=24 mf://@stills.txt
rm /var/www/*.jpg

The third line in this script does something really rather clever it creates a video from all the images you have taken. In order for this to work you will need to install mencoder (sudo apt-get install mencoder)

This is all rather good but wouldn’t it be rather cool if this images were uploaded somewhere. If someone was up to no good and they might want to disable your security camera. My first thought was to use something like DropBox, but unfortunately it is not supported by the processor on a raspberry pi. However I have found copy.com to be rather good for syncing files.

To install just run wget http://copy.com/install/linux/Copy.tgz and extract the file to install the client on your raspberry pi.

./CopyConsole -daemon -u=gary@example.com -p=’mypass’ -root=/home/pi/Copy

This command syncs the folder you have set in -root with copy.com.

The last thing I did was create a php file running on the webserver that allows you to scroll through the different images. I experimented with using javascript to auto scroll through the images but I found the manual way more useful. All the files I have written I have added to my github page.

The Raspberry Pi Adventure Starts

Sat, 11 Apr 2015 20:00:45 +0000

I am currently buzzing with excitement about my Raspberry Pi. I will try to document here what I have done so far.

I decided to buy the camera module along with my Pi. It was straightforward to connect them up, lift the connector behind the network port, insert the ribbon cable, push connector back down.

Next I connected up Keyboard, Mouse, Monitor, SD Card and Power. And my Pi had life.

I was then given a screen full of options to install, Pidora is an OS which is based on Fedora which I have some experience of so I went with that one.

After that was installed I was presented with a standard desktop with various programs installed.

So far I had not plugged in a network cable because that would mean sitting in the hall next to the router, I thought before I do that lets see if wifi will work. I have a usb wifi adapter that I use with my PC, so I plugged that in.

Nothing happened, having some experience of linux I wasn’t surprised. I went to network connections filled in the name of my wifi connection and its password and it connected. WOW! That was easy.

So I then installed webmin so I could remotely administer it and updated it with YUM. I also started the SSH service so I can use putty to login to my Pi from the comfort of my laptop. Even better than that I found a SSH client for my smart phone so I can even control my Pi from my phone!

EDIT: I forgot that in order to get webmin to work I needed to do the following.

mv /etc/redhat-release /etc/redhat-release.bak echo “Fedora release 20 (Heisenbug)” > /etc/redhat-release

OK So now I am able to configure settings on my Pi from a browser or use SSH (Secure SHell) to login to run any command I like.

Now lets lookup how to use the camera. The documentation says that the camera first needs enabling using the command raspi-config. This command doesn’t exist in Pidora. Lets skip it and see if it works anyway. It does! The command raspistill -v -o test.jpg took a picture of my ceiling.

Now how do I look at this picture remotely. SSH is a command line only interface, so I can see a 2Mb file has been created. How about installing a web server on my Pi?

yum install httpd

This install apache which is the default (and very popular) web server.

service httpd start

This starts the web server. Going to http://[IP] where [IP] is the ip address of your raspberry pi will show you the default apache page.

Now if you run raspistill -v -o /var/www/html/image1.jpg it will create your photo on the web server.

Not bad for my first steps, the command above could be scheduled and images taken every hour. There is also options for recording video.

Building something with a Raspberry Pi

Sun, 05 Apr 2015 20:00:45 +0000

Many years ago I spent a lot of time learning about electronics. It was great fun, I can’t remember specific things I built, but learning what resistors and capacitors do was fascinating.

I haven’t really thought about this much as my interests have been concentrated on computers and programming, but recently my interest has been peaked again.

The CodeNewbie and Hanselminutes podcasts have been running a month of stuff about making things and doing clever things with hardware called #Marchisformakers , this began to start me thinking about this stuff again.

Yesterday a friend of mine was showing me what he had been doing with a Raspberry Pi and Arduino. It is amazing what you can do with these tiny bits of technology and I was very impressed.

Now I am thinking what mini project could I play around with and which piece of hardware should I base it on. I don’t know much about the two platforms yet but I have found a blog post that introduces the two. My current thoughts are to start with the Raspberry Pi, it is a UK technology and it uses a linux OS (which I have some knowledge of).

Being a windows developer, most of my programming knowledge is with C# and Visual Studio. To use my existing knowledge I could install and use MONO, which means I can program the Raspberry Pi using C# (see here for more details about MONO on Raspberry Pi).

But deciding on a language to use, doesn’t help me choose a project. I haven’t even got myself a Raspberry Pi yet so I may decide more once I have had a play around with it. Thoughts so far include getting wi-fi to work on the Raspberry Pi or maybe adding a camera and sticking it somewhere and taking pictures, maybe even build myself a baby monitor (might go down well with the wife if its baby related)

Building a CI Server with TeamCity

Wed, 01 Apr 2015 20:00:45 +0000

TeamCity is a Continuous Integration software package which is very easy to install and set up. So easy I managed it over a weekend.

First of all I fired up a VM in Azure. I found that a Basic A1 VM was sufficient so running a CI server doesn’t have to be expensive, alternatively you could always set a Virtual or Physical server on your own network, but that of course takes a bit longer to set up.

Before I get too far you are probably asking what is Continuous Integration. A Continuous Integration server takes your code from your source control repository, it then automatically builds it, runs automated tests against it and can even deploy it to production. The main advantages of this is that you can see the health of you code, you can also test different configurations.

So once you have a server ready, you need to install a few things. First TeamCity , just install with the default settings. My project is a Visual Studio project so I also needed to install Visual Studio (They have a free edition which does everything I need now, Thank you Microsoft) and SQL Server Express Edition.

Once TeamCity is installed the rest of the config can be made via the web interface. To make it easier I pointed a domain at my VM (a CNAME record is probably better as an Azure IP address may change) and opened port 80 in the Azure config. First create a build config and then create a VCS Root, this is where you specify where your source control is. TeamCity supports Github and bitbucket and probably many others. You can specify which branch you want to checkout so you can test out the latest features you have written.

Next you need to configure the steps needed to build your project, whatever you do in Visual Studio can be configured here. You then need to configure how often the build should run, it can be scheduled at specific times, this is how I remember CI servers being configured but now you can also have builds being triggered after commits to source control, very clever.

That is just about all I have configured so far. My project has a few NUnit tests written for it, these are scheduled to run after every build. In my case 94 of them are currently failing so I now need to learn how my tests work.

There are additional deployment steps that can be configured but I have not investigated these yet. Notifications about what CI is doing is also possible usually by email but there are other options available.

Monitoring Screens

Wed, 25 Mar 2015 20:00:45 +0000

We all know that it is important to monitor your servers and services, so you can spot issues before they become problems. I personally have spent a lot of time configuring nagios to email me about issues and I have recently been configuring various different alerts in Azure.

My old boss has this idea that I should have a big monitor screen displaying all the vital stats of my servers and services, I personally disagree with this idea and think that notifications on my phone and email alerts are sufficient. He will no doubt correct my thinking when he reads this, but I believe part of his thinking is to make the monitoring of your infrastructure move visible and make it obvious to anyone that walks past that you have your eye on everything.

For the purpose of this blog post lets assume he has convinced me and I have convinced my actual boss to spend money on the required technology to do this (No easy feat). What exactly would I display on this screen?

I have Google Chromecast that I use for streaming various things to my TV, this is a relatively cheap bit of technology that could allow a TV or monitor to display a web page with the required stats displayed.

The two main sources of information that I want to display are New Relic for monitoring my azure websites and Nagios for monitoring my internal servers. New Relic allows you to easily export live performance data as iframes so I quickly threw together a web page full of these graphs. However if you have a static screen on the wall you don’t want to have to scroll to see different information so I needed to come up with another way to display this information.

My first thought was a slide show. There are lots of javascript scripts that cycle through a series of images like a slideshow, this could be adapted to cycle through a series of iframes and display everything I want.

My script goes something like this and requires jquery as well as javascript. First of all the script waits for the page to load completely with the ready function, it then defines the urls which will be put into the iframe one at a time. It than counts the number of urls you have. It then loops through changing the contents of the src attribute in the iframe every few seconds, in my example it changes every 9 seconds but once this is used in production you may want to increase this.

<script type="text/javascript">
$(document).ready(function(){
  var locations = ["URL1", "URL2", "etc"];
  var len = locations.length;
  var iframe = $('#frame'); 
  var i = 0;
  setInterval(function () {
    iframe.attr('src', locations[++i % len]);
  }, 9000);
});
</script>

Now what information wants to be included in a script like this? Showing too much performance data can almost be as bad as not doing it at all as problems gets drowned out in the noise. For me I have performance of my websites, followed by Nagios problems, followed by the azure status page, followed by memory usage of all my servers and lastly showing number of connections to my databases. Another question to consider is what time scales do you want to graph over, too long and you don’t see what is happening now, but too short and you may only worry about an intermittent issue?

Networking Event

Sun, 22 Mar 2015 20:00:45 +0000

I recently went to an event called ITBoss organised by Core Com Consulting . It was really good, helped partly by a free bar and great food.

The event was hosted in a pub in central Leeds, so I rushed to the train station from work to get there. And yes I was the first to arrive (Why do I always do that?). The event was targeted at IT Leaders and Managers and was limited to about 20 people.

To begin with I just chatted to the hosts and other people as we waited for everyone to arrive. Once we got going we had a short presentation about the need for Strategy. Interestingly the next day I suggested a strategy meeting with my director, so something must have sunk in from this.

The main gist of the talk was that IT leaders need to pop there head up from day to day work to look at where they are going, and this should be matched with the overall direction of the company. I agree with this, recently I have been thinking of what direction our infrastructure needs to go in, it is currently just in my head, but at least that is a start. However our company directors are poor at sharing their long term vision, so it is hard for me to know where the company is going. I need to step up and challenge this behavior, so I can deliver the changes that are needed. Not easy but that is what the best IT leaders are doing.

After this we broke for food, which was very nice. I had a chat with a couple of people. I found it interesting that one gentleman was surprised that I wanted to get into development. I think I understand why, most of the people in the room where senior managers, while I share their job titles, I don’t share their salaries. He saw development as a demotion, I see it as a promotion.

After food we had some discussion questions. Mine was the first, How to convince others of the need for change? Some interesting ideas where presented, but it is the need to work closely with others, find out their requirements which is the difficult part not anything technical. The other topics of discussion where relating to recruiting people with skills that you have no knowledge of and around the topic of security and privacy in this modern age. Both were very interesting with lots of viewpoints expressed.

I enjoyed the evening and hope to go to another at some point. Previous events I have been to have been technical, but lacked the networking that this one had. While I am not very good at networking, at this event it was easy to have a conversation or two with people.

Source Control Fail

Mon, 16 Mar 2015 00:00:00 +0000

Every developer uses source control, it is a great tool for keeping track of changes to your code, seeing who has done what.

However I keep messing up, my use of it. I am fairly disciplined when creating new features, all my changes will get committed to source control and when I am happy I will deploy these changes to the live system. But as soon as there is a bug, especially one where the client is chasing for a fix, I will deploy a fix as soon as humanly possible on to the live system.

At first glance there is nothing wrong with what I have described but what happens the next time I deploy a new feature. Yes the bug the client was complaining about gets deployed with the new feature as the fix was never committed into source control. The client gets angry as the bug he was screaming about is back again.

Every change you make to the system MUST be committed to source control. If it isn’t that change will look like it never existed. I have worked with source control for over 5 years why do I keep making this rookie mistake over and over.

Troy Hunt has a blog post about the 10 commandments of using source control. His number two commandment is “If it’s not in source control, it doesn’t exist” He talks more about code you have written being not saved into source control, but the principal is the same for my example as his.

There are ways to automatically deploy from source control, however most of the time you don’t want your live database being rebuilt because you fixed a typo. Additional steps will need to be implemented and there is still the chance that you might want to bypass these steps to fix the urgent problem. The only way past this problem is for you and everyone on your team to be disciplined and only ever commit to source control first, and only after that deploy live (either automatically or manually)

Azure Traffic Manager

Thu, 12 Mar 2015 20:00:45 +0000

I have spent most of the day tweaking my Azure websites. Lots of fun!

Last week unfortunately Azure had some problems and many websites that were running in the North Europe data centre were unavailable for several hours. And you guessed it my websites were hosted here.

All hosting providers are going to have downtime from time to time and this is just something you have to take on the chin. The important thing to do in times like these is communicate with your customers about what is going on and that you are doing everything you can to restore service.

However Azure has some amazing features that you can configure to help manage when downtime occurs.

Azure is Microsoft’s global cloud platform. And it really is global, there are data centres in North Europe, West Europe, Brazil, Japan, two more in Asia and five in the US. In the event of problems it is highly unlikely that more than one of these would go down at once. If all of these are unavailable, I expect the planet earth is facing some kind of cataclysmic event and the fact that my website is down is not a priority.

To take advantage of these multiple data centres, Azure has something called a Traffic Manager.

Traffic Manager has various settings but I am using it in failover mode. This means that if one website goes down, the next one is used.

All you need to do is create a traffic manager, add two or more websites to it (called endpoints) and choose a page that needs to be monitored so Azure knows which websites are up and which are down.

If you are using SSL or custom domain names, there are a few extra steps you need to do. Your custom domain name needs pointing at the traffic manager, not the individual websites. The websites themselves have three domain names, the traffic manager address, the azure address and the custom domain name. The SSL certificate can then be assigned to each website that you have added to the traffic manager.

That was easy wasn’t it, and now if a website goes down traffic manager will use the next one. While testing this, the transition to the next website was almost immediate. I did notice that if you had a browser showing the website open during a problem you sometimes got an error page, I think this was probably due to browser caching, reopening a tab or browser fixed this issue.

Tidying my desktop

Wed, 11 Mar 2015 20:00:45 +0000

My desktop is always a mess. I constantly download files there and forget all about them.

Every now and then I copy files into sub directories, so my desktop looks sane for a day or two before it gets out of control again.

Why don’t I write a script that I can schedule to do this for me. Then my desktop will always be tidy.

I have written a few simple batch scripts, but of course the best scripting language out there at the moment is PowerShell. Lets use that.

Windows provides a nice little utility for writing scripts called the Windows PowerShell ISE, so let’s start by loading that up.

PS has lots of help included to help you, just run Get-Help [name of ps command]

To move files you can use Move-Item which works very similar to copy, specify source and destination. In my case I moved files based on their file extension.

Move-Item *.pdf folder

Now all I need to do is schedule this script to run either every day or so, or maybe every time I login or switch my computer on.

PowerShell can do lots more interesting things which hopefully I will blog about soon.

Database Deployment

Thu, 05 Mar 2015 20:00:45 +0000

One of my jobs is adding extra features to our internal databases. There is a fair amount of risk in doing this. I could make a change to one of the queries in the database that stops the rest of the company using it as they need to. This could be a costly mistake, almost as bad as the database server being inaccessible.

One way to reduce this risk is to test, test and test the database. First I take a backup of the database and make my changes to that. I can do anything I like to my backup database without worrying that it will affect anyone else. Once I have finished making my changes and I have tested that everything is working exactly as I want, I need to apply my changes to the production database.

Until recently I made a note of every change that I have made and then made the same change to the production database. But what if I forget to apply one of my changes to production, I have a broken database on my hands, disaster!

But there is a better way to track database changes. SSDT or SQL Server Data Tools.

Create a Database project with Visual Studio, you can then point this at your live database. This will scan the database and save scripts for all the Tables, Views, Stored Procedures, Triggers and any other object in the database. This doesn’t import the data, just the structures that contain the data. So no data protection issues or problems because your database is terabytes in size.

Now comes the really cool part, Visual Studio allows you to build a copy of the database from these scripts. All you need to do is edit the scripts and test like you did before. Once you are happy with your changes and everything is working as you want, you can point your visual studio project back at your production database and it will magically deploy all your changes, or if you are as paranoid as me about breaking things, it will create a script that you can thoroughly analyse before running on your production database.

I do not know why I never thought of setting this up until recently, it will reduce the number of times I roll out my changes but forget to setup permissions, (these can be scripted) or forget that tiny change I did weeks ago but never rolled out.

And of course all the changes that you make can be saved in source control, so easy to see what changes have been made and by whom.

This process even works with complicated database structures, in my case I needed to setup three database projects that depend on each other. Dependencies can be setup, and the test databases can even be built with different names for testing on the production server, (although I wouldn’t recommend it!)

He’s really not dead as long as we remember him – Leonard Nimoy

Sat, 28 Feb 2015 20:00:45 +0000

n the film Star Trek 2: The Wrath of Khan Spock sacrifices himself to save the Enterprise from destruction. What follows is a very moving funeral scene and his coffin is launched into space. McCoy tells Kirk that he is not dead as long as we remember him.

Yesterday the actor Leonard Nimoy passed away and millions of trekkies around the world are taking comfort in those words.

Reading social media it is clear that he affected the lives of millions of people with his portrayal of Spock. I have seen tweets from NASA and astronauts saying how they were encouraged by Star Trek. The chip manufacturer Intel even tweeted about his death. I am sure there are many in the science and technology space that could say Mr Spock encouraged them in some way.

The portrayal of science and technology in Star Trek had an affect on me. I studied Mathematics, Chemistry and Physics at school, I later did a Physics degree. All because the spark of curiosity about how stuff works was encouraged by a TV show. I remember one time while studying physics learning about the hysteresis curve, then Star Trek the next day mentions detecting changes in the hysteresis curve might solve some problem. Wow the stuff I am learning is in Star Trek!

Later in my life I got involved in working with computers, and as science officer Mr Spock was the first expert in computers I knew.

But Mr Spock was half Vulcan and suppressed his emotions. There have been times in my life where I have tried really hard to suppress my emotions. I can think of a time when a family member has died and I needed to shut myself in a room, similar to Spock in Naked Time, and wrestle with my emotions. In recent years I have had a more Data approach of exploring my feelings and my humanity as a Vulcan lifestyle of suppressed emotion is not healthy.

Thank you Leonard Nimoy for all that you gave us, although I never met you I can continue to watch and enjoy your work, Live Long and Prosper.

Source Code for Game of Life

Sat, 21 Feb 2015 09:00:45 +0000

Last week I blogged about the Game of Life.

Well it took some searching through dead hard drives and old USB storage but I found the program I wrote, and better than that I have figured out how to turn it back into source code.

The file date is November 2004 and the source code has no comments so I don’t know what was going through my head when I wrote it, or even what some of it does.

To view the source code download Life.src , to download the finished program download Life .

I am actually surprised I was expected the code to be much worse than it was, I am not saying it is good, but I can see more than one method and I can’t see any code that obviously is repeated. I can see some terrible variable names. Note to future self don’t use tmp676_674!

Assuming that I wrote this in 2004 what would I have been doing then? It was before I started working in IT. (Hard to imagine I know!) It may even have been before I started living with Keith, before he recruited me for my first IT role. I think I was probably working as an administrator for Defra at this point.

If we put 2004 in technical terms, it was before I joined facebook, Office 2003 and XP would have been installed on my PC, and probably felt brand new.

Cloud Computing

Thu, 19 Feb 2015 20:00:45 +0000

I work for a company that provides a cloud computing website, but what is Cloud Computing?

Wikipedia defines it as “Cloud computing involves deploying groups of remote servers and software networks that allow centralized data storage and online access to computer services or resources.”

Erm. What does that mean? Well in simple terms it means storing you data on a remote server. Any time you submit information onto a website, you are submitting your information to the “Cloud”

But there is more that the Cloud can do for you. The search giant Google, which also develops the android mobile phone operating system, have developed a range of cloud computing products. The Google Drive product allows you to upload and even create files, these files are saved into the Cloud, once in the cloud they can be accessed from any device that has an internet connection. No longer do you run the risk of your favourite file being on a computer that won’t turn on or being on a dead hard drive, save it to the cloud and you will always have access to it.

Google isn’t the only one that does this, Microsoft have OneDrive which they are incorporating into the heart of their operating system, Apple and many other companies have released similar products.

This age of cloud computing allows a clearer separation between your data and your device. It won’t be long before whatever device you are using your files will be there to be worked on. You are in the office working on a spreadsheet, you save it to the Cloud, you can then continue working on it with your tablet while on the road. Just before bed you want to check if you added something to that doc, so you quickly check your phone. How cool is this!

But if I put my system administer hat on, Cloud computing goes far further than that. Microsoft Azure allows many of the services you would typically have running on your server, saved into the cloud. This has many advantages and allows you to concentrate on other problems while leaving the cloud to look after these services.

Choosing a Blog Theme

Sun, 15 Feb 2015 20:00:45 +0000

You may have noticed that my blog has a new home. It can now be found on its own domain http://www.funkysi1701.com . This change not only gives me more flexibility to make any change I like, it also feels good to have a website again. If you had followed my blog, do keep following it at the new location.

I have also been playing about with giving the site a new look, hope you all like it. I have gone for a more image focussed front page. No doubt in a few weeks I will be bored and change the look again.

I have recently been reading about blogging and one suggestion is to have a clear theme, or idea of what the blog is about. I am not sure my blog has a clear theme.

When I started blogging I was more interested in blogging than thinking about what I was blogging about. The vague theme I worked off was to talk about what I do, to answer the question, “What do you do?”

I think this is a fairly good theme, but it could probably do with some refinement. The best themed blogs are specialist or about something specific. As I am generally learning about development, rather than doing development I think this should be part of what I concentrate my blogs about. A minority of my blogs can continue to be about the other things I spend my time doing, as it’s always good to have a break from the main “meat” of the blog every now and then.

The people who read my blogs are split between technical people and non-technical people, so I would like to continue to explain what I do in non-technical terms. This is an essential skill for a developer (or SysAdmin) as you will always need to explain what you are doing, or discuss requirements. It is not an easy skill and like development itself, the more I do it the better at it I will become.

So watch this space for the blogs to come. I know I will be working with Azure over the next few weeks so maybe some blogs about that, maybe some more SQL blogs or maybe about what I learn next.

Ten Forward Episode #135 – Anti Firbob is Back or Simon’s Desert Island Trek

Fri, 13 Feb 2015 20:00:45 +0000

The podcast I recorded the other week has been released, why not have a listen by heading over to trekmate.org.uk and comment on their forums

This week Sina, Chris and Paul welcome Simon Foster, Trek Mate’s News Editor, to do discuss his Desert Island Trek choices. Paul becomes Firbob, or AntiFirbob, or…something like that. Boo hiss Firbob. Every one listen and decide – Simon or Firbob?

You can reach Simon here:

@funkysi1701

Complaints to @firbob1

You can reach the rest of the gang here:

@queenkatblue , @chriscantfly , or facebook.com/TenForwardPod

Game of Life

Wed, 11 Feb 2015 09:00:45 +0000

Many, many years ago I tried to learn Java. I didn’t get very far but one of the things I built was The Game of Life.

Unfortunately I can not find the code I wrote, probably a good things as it is probably amusingly bad.

Now you are probably asking what is the Game of Life ? The game of life, sometimes called cellular automation, is a simulation of cells being born and dying. It is sometimes called Conway’s Game of Life after the British mathematician who devised it in 1970.

The game has several rules:

Any cell with less than three neighbours will die on the next turn
Any cell with three neighbours lives on to the next turn
Any cell with more than three neighbours will die on the next turn
Any dead cell with three live neighbours will be born on the next turn

Now I had not thought about this game until the other day when I came across a blog and a github page which sparked my interest again. This version is written in javascript, but the blog mentions a desire to build this program in many different languages. I want to build this with visual studio in C# as it should provide a good training exercise.

There are several problems I need to overcome to achieve this, I am currently trying to work out how to “draw” the cells. My current thinking is that maybe I can code it using characters to display the cells and then replace with pixels once I have the logic correct. Well lots for me to learn!

Great British Code Off

Sat, 07 Feb 2015 20:00:45 +0000

The Great British Bake Off and The Great British Sewing Bee are very popular TV shows. Each week the contestants take part in a baking or sewing challenge and the judges choose the best and the worst, with the worst having to leave. This is very entertaining as you watch the drama between the contestants and the reactions as the judges tell them they are rubbish.

Could this simple format be applied to computer programming? I think it could.

The only developers I know argue about the best way of tackling different problems, despite being friends. I am going to assume that this is typical behaviour and not unique to them. So for this TV show, we will split the contestants into pairs, that way we would get the added drama of how well they work together.

I am still undecided if the pairs stick together each week with one pair leaving or if we randomize the pairs with one person leaves. The second option would cause there to be a group of three every other week, not sure if this would be a benefit or a curse for that group.

My friends who are experienced programmers have agreed to be the judges, they both have experience of hiring and firing people, know what to look for in good code, and never agree so should be entertaining enough.

On the bake off the contestants don’t really know how well they have done until the judges cut into their cake and take a bite. For the code off we could not compile the code until the judges run it, would make coding extra difficult as if the contestants are anything like me, I run my code every five minutes to see what affect that change has made.

So what challenges can we come up with?

Web Design week
Stored Procedure week
Mobile App week
Loops
Logic problems
Bugs weeks
Performance issues

I think we are onto a winner here. What does everyone else think?

Laziness

Wed, 04 Feb 2015 20:00:45 +0000

I am lazy, I won’t try and deny that. When my alarm goes off in the morning, I will snooze it for twenty minutes or so before getting out of bed.

In my work my laziness continues. Remote Desktop (or RDP) is probably my number one laziness tool. For those that don’t know RDP allows you to connect to another computer and access it like you were sat in front of it. So I can be sat at my desk and RDP into any other computer in the office including any server. However this laziness tool does sometimes require a bit of effort sometimes, powering on the target computer, logging on locally, making sure the user account you are using is allowed to use RDP.

Writing a script is another example of a laziness tool. I often get asked to do tedious and long-winded task, because I am lazy I will go out of my way to learn how to write a script to do this, so that I can run this script and do this long-winded task in a matter of seconds. There are loads of different types of scripts from database SQL scripts, to PowerShell scripts that can do almost anything on your server.

PowerShell is something Microsoft are really pushing at the moment, you can even write scripts to create new user accounts, so no longer will you have to remember to tick that tickbox for every new user account. And because PowerShell is part of almost all MS technologies you can link Exchange to Active Directory and you don’t even have to remember the right syntax as PowerShell has a built-in help command to tell you how to run that useful command.

As I am now moving into a more developer role can I continue to be lazy? I sure can. Code should be written once and reused as often as possible and this is one of the features of OOP (Object Oriented Programming).

It can be as simple as creating a master page so you don’t need to recreate the same code on every one of your webpages. Or every time you find yourself rewriting the same code again, you plug it into a method so it can be called again and again.

But of course the ultimate way to be lazy is of course get yourself some staff and spend all day getting them to do everything. If you are lucky they may even try and adopt some of these lazy ideas.

Podcasts

Tue, 27 Jan 2015 20:00:45 +0000

For a while now I have been listening to podcasts. For those that don’t know a podcast is an audio file that you can download and listen to, usually about a topic you are interested in. The topics that interest me are obviously Star Trek, Science and Technology and IT stuff. I mainly listen to them in the car when I am driving, but sometimes it will be at home when doing jobs around the house.

I think the first podcast I listened to was Trekmate . Trekmate as the name suggests is about Star Trek, it’s a few guys talking about Star Trek. It’s very similar to those conversations you have in the pub with your mates. Trekmate do lots of podcasts, one for almost every flavour of Star Trek out there, I don’t keep up with all of them, but the ones I listen to I really enjoy.

My favourite podcast that is work related is RunAs Radio . If you are a SysAdmin or have an interest in servers or IT you should listen to this. The host Richard Campbell interviews IT experts about a particular piece of technology, one week it will be Exchange, the next Security, and then SQL Server. Often I will learn something that I will want to try out, some weeks I won’t learn something specific but will give be an insight into which way IT for businesses is going. I think it was probably only after listening to RunAs Radio that are started having more of a long term plan for my IT infrastructure at work.

As I have blogged about before I am learning Development so obviously I listen to .NET Rocks . This is hosted by Carl Franklin and Richard Campbell (yes the same guy that does RunAs Radio) who talk to developers about code, learning and IT from a more Developer point of view. This is released 3 times a week so is a bit hit and miss sometimes, but most weeks there will be something relevant to me or worth listening to.

A few days ago I did something for the first time, I was a guest on one of the Trekmate pods, TenForward. I reviewed 5 Star Trek episodes. This was a bit of a leap for me as an introvert I don’t like to be the center of attention and I certainly don’t like how my voice sounds recorded. Hopefully when released people will like it and I will do lots more of these in the future. Once it has been released I will put a link up here so you can all have a listen.

Pair Programming

Mon, 19 Jan 2015 20:00:45 +0000

So I have been working through the challenges on the freecodecamp website and one of the unique things that this site does is give you the chance to program with another person.

I am an introvert and if you were to ask me to pick up the phone and ring someone I will procrastinate a lot before doing it (if I do it at all), I also am not good at speaking with strangers but I have been enjoying the pair programming I have done so far.

I have only pair programmed a couple of times so far, so I am no expert. The first time was with someone from North America, it was weird to begin with talking to your computer and hearing an american voice respond, but you soon forget that as you concentrate on the problems you are working on.

The second time was with someone from the Netherlands. This time we tackled some trickier problems. Both times I took more of a back seat but I think contributed useful stuff and I certainly learnt a lot from the other person.

I think that is the key thing about pair programming, you learn so much about how to look at a problem from another angle, or learn about a function you haven’t heard of before. If you are reading this and it was you that I pair programmed with a big thank you for spending the time with me, I learnt a lot.

I was trying to pair program again tonight but technical problems prevented me. One problem I have found is finding a partner, was hoping to do some yesterday but no one else was around to partner with.

I would be interested to learn what companies use pair programming. I imagine a pointy haired boss would say why pay two people to do one job, when you could get them working on different things. However if you want to build up your team and give them loads more skills then pair programming would be great. I also think there are probably some problems that are crying out for pair programming.

Javascript progress

Fri, 16 Jan 2015 20:00:45 +0000

I got excited earlier this week as I started playing around with the javascript I have been learning.

I thought I would try and create useful javascript that could be applied to an online filestore. Somewhere that you could upload, delete, edit files and folders and various options to do with your files would popup via javascript or jquery code.

So far I have created a static HTML page which lists a few dummy files. I have added a button which adds a new row into my table and I have used jquery to highlight any selected files and also when the mouse passes over.

If you want to follow my progress I have put my files on github . It’s very basic stuff and I have already got a few lines I would like to refactor but not bad for an evenings work. I have lots of feature ideas to add to this page which should help me learn the basics.

The trouble with learning all this javascript is that I am starting to get all my programming languages confused. Earlier this week I started writing a for loop in javascript, only problem was I was working on an MS Access ADP (Don’t ask!) which requires the code in VBA. That is never going to work. Oh well.

If anyone knows good ways to stop myself getting my programming languages all confused do let me know.

Hello World!

Sat, 10 Jan 2015 20:00:45 +0000

The first program anyone writes in a new language is often really simple and just displays the text “Hello World!”, below are a few examples from languages I have knowledge of.

Console.WriteLine("Hello, world!");

<html>
<head></head>
<body>
<p>Hello, world!</p>
</body>
</html>

php echo 'Hello, world!'; ?>

MsgBox("Hello, World!")

SELECT 'Hello, world!'

Or maybe

SELECT * FROM table WHERE name = 'Hello World!'

'Hello, world!'

console.log('Hello, world!');

PRINT "Hello, world!"

Turns out I know quite a few programming languages. And yes there have been times where I need to stop and think about which language I am writing in.

My Year of Code

Fri, 09 Jan 2015 20:00:45 +0000

There is no escaping the fact that we are in 2015 now. My new years resolution is to do more coding.

Over Christmas I thought I would have a go at http://www.codecademy.com I completed the javascript course and I was just wondering what I should try next when I saw a tweet about http://www.freecodecamp.com

Since then I have been addicted to doing the exercises that they have, almost everyday I have made a bit of progress.

But this site is not just about teaching you how to code, it has an active forum where you can ask questions and understand more about what it takes to become a developer.

Before I started these exercises I didn’t know much javascript or jquery but I am starting to learn quite a few commands that hopefully I can put into use at some point. Whatever development I end up doing I am sure I am going to need javascript at some point.

One of the tips I have learnt is that you need to practise daily. Did you ever use flash cards to help pass exams? Well there is an electronic version of them now called Anki. Just add a question/answer pair to your deck of cards and review them each day, the ones you find difficult you review more often than the ones you have learnt.

Hopefully I can keep this up and later in the year I will be well on way to writing lots of useful code.

Looking back at 2014

Wed, 31 Dec 2014 20:00:45 +0000

As today is New Years eve I thought it would be a good chance to look back over the last year.

At the start of the year I finally got agreement for the installation of a 30MB leased line. This was an amazing achievement as I had been really struggling to keep everything working on our ADSL connections and the complexity of the connections was starting to cause problems itself.

However it was not a smooth process, the site survey revealed a substantial installation cost would be required, however the Council was about to launch a voucher scheme that would give us £3000 off. So most of this was just waiting around for the Council to get things going. By April we had it installed and I have been very happy with the ISP we chose, York Data Service (YDS).

Also during April this year was the end of support for Windows XP and Office 2003. This didn’t impact us too much, mostly due to my constant nagging of the directors to replace our oldest machines. We now only have a couple of machines still running XP and no machines running Office 2003. Most machines are now running Windows 7, I have one Director running Windows 8 for presentations and a few surveyor tablets running windows 8 as well. Like many businesses I am waiting for Windows 10 next year before doing any more substantial replacement of OS.

In May it was decided that I would move more into development work, so I got a new desktop PC and Visual Studio. I am very happy with my new machine (subject to dell fixing a problem later today) So far I have only dipped my toe into development work, this is largely due to needing more time to build up others to do what I do.

In June I made another huge achievement. I got agreement for a new server. This was our first Server 2012 server and allowed us to make huge strides towards virtualizing our services. Once we had this server up and running I was able to decommission two of our Windows Server 2003 machines well before it goes out of support next year.

I am a huge fan of this new server, I have only used a fraction of its memory. Hyper V is really great and I hope to make progress next year into HyperV replication.

Also during 2014 we made huge progress with our clouds software. Within the last few weeks it is running on the Azure platform which will not only make it far easier to expand and create new features it will provide us with cost savings. I am looking forward to learning more about Azure as I get deeper into development.

Wow I achieved a lot didn’t I, on top of this I hired a new IT person and trained him up (still more to do), I have also fixed lots of printers, websites, servers (had a server motherboard die!), and made hundreds of changes to our internal databases.

I don’t know what is in store for 2015. I expect a lots of the same with hopefully more focus on developing in Visual Studio, training myself and my department and getting the last few tweaks into our internal servers.

I would also like to thank everyone that I have worked with over the last 12 months (you know who you are), as you made a lot of what I have just described possible.

Happy Christmas

Wed, 24 Dec 2014 20:00:45 +0000

I saw this tweet on twitter.

He’s making a database,
He’s filtering twice
SELECT * FROM customers WHERE behaviour = Nice
Santa Clause is Coming to town.

This started me thinking surely in a normalized database structure behaviour wouldn’t be stored in the customer table, so I propose the following change.

He’s making a database,
He’s filtering twice
SELECT * FROM customers WHERE EXISTS (select * from behaviour where behaviour.CustomerId = customers.Id and behaviour.Type = Nice)
Santa Clause is Coming to town.

Happy Christmas everyone, hope you all have restful holidays.

Software Developer Quiz

Fri, 05 Dec 2014 20:00:45 +0000

Moving house, lack of internet and lack of inspiration has caused a lack of posts recently but hopefully more to come. Been doing some filing and found an old software developer quiz. Thought I would have a go.

Many thanks to Keith for originally writing the quiz.

Questions:

Write a function that determines if a string starts with an upper-case letter A-Z
Write a function that determines the area of a circle given the radius
Add up all the values in an array of integers
Given a table called “Nodes” with the following structure and sample data (see below after questions): …where ID is the primary key,andParentID references ID, complete the following:
- write a stored procedure to return all nodes beneath a given node ID
- describe how you might write a query to return all nodes at any depth below a given node ID (i.e. recursively)
Write a function to get the prime numbers up to 1,000,000
You’ve been given the following code to review (below table)– what comments would you give back to the developer?

ID	ParentID	Name	Type	Depth
1	NULL	My Documents	Folder	0
2	1	My Pictures	Folder	1
3	1	My CV	Document	1
4	2	Photo of me	Document	2

CREATE PROCEDURE GetNode
@NodeId INT
AS

DECLARE @ID INT, @ParentID INT, @Name NVARCHAR(255)
DECLARE @Type NVARCHAR(20), @Depth INT

SELECT @ID = ID FROM Nodes WHERE ID = @ID
SELECT @ParentID = ParentID FROM Nodes where ID = @ID
IF (EXISTS(SELECT NULL FROM Nodes WHERE ID = @ID AND Name = NULL))
SELECT @Name = ''
ELSE
SELECT @Name = Name FROM Nodes WHERE ID = @ID

SELECT @Type = Type FROM Nodes WHERE ID = @ID
SELECT @Depth = Depth FROM Nodes WHERE ID = @ID

SELECT @ID, @ParentID, @Name, @Type, @Depth

My Answers:

static bool GetUpper(string var)
{
  if (char.IsUpper(var[0]))
  {
    return true;
  }
  else
  {
    return false;
  }
}

static double AreaOfCircle(int radius)
{
  double area = 0;
  area = Math.PI * radius * radius;
  return area;
}

static int SumArray()
{
  int[] MyArray = new int[10] { 1, 2, 5, 12, 4, 9, 8, 18, 9, 6 };
  int Sum = MyArray.Sum();
  return Sum;
}

create procedure getnodes
(
  @node int
)
select * from dbo.nodes where parentid = @node

For a recursive query I would write something along the lines of, but it would need to be customised depending on the depth, eg more joins for higher depths

select * from dbo.nodes n1
join dbo.nodes n2 on n1.id = n2.ParentId
join dbo.nodes n3 on n2.id = n3.ParentId
where n1.parentid = 4544054

static void prime()
{
  Console.WriteLine("Prime: 1");
  for (long i = 3; i <= 1000000;i++ )
  {
    bool isprime = true;
    for (long j = 2; j <i; j++)
    {
      if(i%j==0)
      {
        isprime = false;
        break;
      }
    }
    if (isprime)
    {
      Console.WriteLine("Prime: "+i);
    }
  }
}

No Brackets around parameters, @NodeId parameter never used, select @id = id from dbo.nodes where id = @id is pointless as same id that is passed it being set, Name = NULL should be Name is NULL, no from specified in last query. There are probably more issues as well.

In the Zone

Sun, 02 Nov 2014 20:00:45 +0000

On Friday night I had the urge to fire up Visual Studio and tomorrow (Monday) I move house.

What’s with that? Why am I in a programming mood when I have boxes to pack and shelves to take apart?

For the last few weeks I have been too busy with other things to do much programming or when I had the time I felt more like relaxing than learning anything new.

How do I get into the programming zone, what was it about this particular weekend that made me want to? Was it just the fact I had no time to concentrate on it, or is there some more useful factor that controls my desire to code?

I can think of a few factors that I should make note of.

Peace and quiet – Friday night I had the place to myself so I could concentrate and do whatever I wanted.
Ideas – Before I started I had some ideas, why don’t I try doing x. Once I have a coding idea I need to investigate it and see where it goes.
Break from routine – Friday night was the start of a whole week away from work, this gives me more time to forget about the daily stress of the office and think about other things.
Goals – Work annoyed me on Friday and if I want to achieve my goal of working in development more then I need to work at it.

I am not sure how I can put these into practise so that I spend more time on development but we will see. One thing I want to achieve in my new home is a quiet place to concentrate and think without distractions, I have a few options which I will investigate over the next few weeks.

Anyway need to get back to packing boxes.

If you had £5000 to spend on technology what would you get?

Sun, 26 Oct 2014 20:00:45 +0000

I listen to the dot net rocks podcast and they ask this question every show. (If you don’t listen then you should!)

I don’t know what my answer would be as there are so many gadgets and cool things that I like the sound of.

I might get a surface pro 3, it’s a tablet with attachable keyboard that’s more powerful than my current laptop. But at £600 it’s expensive so maybe not. Maybe instead I would install a SSD into my existing laptop to increase disk performance.

I’d like a new car with lots of gadgets built-in. A friend gave me a ride in his new BMW and it’s nice. Bluetooth so I don’t have to have an audio cable to listen to podcasts. Parking sensors so no chance the wife will prang it. Built in satnav etc

While we are on the subject of cars eventually I’d like a google self driving car as I think Google is a better driver than me. Think about it I drive to a friends I have a drink I sleep in the back seat as my car drives me home. Once all cars are automated I believe the roads will be much safer.

The last thing I would spend on is an intensive training course on development, I would really like to get further in dev and more training would really help. Specifically something that would help solve my current dev problems. I am sure it is possible for me to learn dev with self-study but that requires a discipline I don’t think I have, I easily eat my time up with my current job, wife and other interests.

I recently had £6500 to spend on improving my employers IT infrastructure. That was really hard. It sounds like a lot of money, but it really isn’t.

Once I started looking at the problems that I wanted to solve and what hardware I wanted I quickly used it all up and that was before I factored in license and software costs. In the end I just had to order what I wanted and hope that on a future date I could spend on other areas. I got a super powerful server that I could virtualize almost all our existing physical servers with it I needed licences, which left me just enough to get a NAS for file storage.

Interview questions

Thu, 23 Oct 2014 20:00:45 +0000

I have done a few interviews from both sides of the table, I’m not very good at either but thought I would have a go at answering my favourite questions.

Describe an IT disaster and what you did to turn it around?

The question is all about the turn it around bit. I’ve had lots of answers that emphasise the problem rather than what the candidate did to turn things around.

My answer would be: on a Saturday I was rearranging the server room and when I came to turn on our main file server and pdc it wouldn’t boot. My plans for the weekend went out the window it was all about getting this server back up. I tried the usual unplug everything and reconnect still nothing. I rang a friend to get a second opinion and between the two of us we formulated a plan of action. I then rang my director to tell her that I was having problems and what I was going to try. (keeping people informed is an essential skill)

So the server in question was getting power but nothing was happening during boot. The motherboard had died. What I needed to do was connect the raid card to another server so I could copy the data from it to our Nas drive (luckily I had an upgrade plan and this failure had just accelerated it)

A few hours later the data was copying and I could breathe again. I told my director that I had fixed things but there would be minor issues on Monday.

Describe your strengths and weaknesses?

It wouldn’t be an interview without a strengths and weaknesses question. As an interviewer I want at least one weakness and I want it to actually be a weakness.

My strengths are my problem solving skills, I can look at a problem and investigate what is going on and find a solution. If its a technology I haven’t used before I can read up about it and find out how it works and then use it to solve the issue.

My weakness is my interpersonal skills, I am much better at analysing a computer issue than figuring out why one member of my staff is not performing. This is an area that is improving over the last few years I have taken on more responsibility over people and I am learning more about getting the best out of them.

Technical Questions

I have never included technical questions in one of my interviews until recently and I think it is a good way to gauge ability. I often come out of an interview unsure how well that person would perform under the stresses of my job. If I have something down on paper it is a good start.

But the personality that comes through during the interview must also be considered as most knowledge can be taught.

A good technical question I have used in the past is write a paragraph explaining DHCP. This question illustrates what their knowledge is like for an essential technology, but it also indicates what their writing style is like. Could the passage they have written be given to a Director or Client?

Good IT people need both technical knowledge and the ability to communicate at all levels, I still struggle to know if someone is good or not and even sometimes if I am any good.

To boldly go where no SQL has gone before Part 2

Tue, 21 Oct 2014 20:00:45 +0000

Let’s continue looking at a database schema for storing details of every Star Trek Episode. If you are new to databases, a schema is just the design of the structure of the database.

We have three tables, Episode, EpisodeWriter and Writer. See my last post for more details of these. It has been suggested that a slight change to this structure would enable storage of more of the creative staff.

Lets rename Writer and call it Credit, and rename EpisodeWriter and call it EpisodeCredit. Now any creative staff member involved in an episode can be stored in the Credit table. Lets alter EpisodeCredit and add an extra column called CreditType. CreditType is just a text field that stores the role that creative person had on that episode, it can be anything from Director, Writer, Actor, Science Consultant etc.

We now have the ability to store information relating to the episode in the episode table and any creative people in the credit table. What else can we add to the database? How about a table that can be used to record when an episode was last watched. I am probably weird but sometimes I want to watch a Star Trek episode that I like but I haven’t watched in ages.

The last watched table is really simple and just have a datetime field and episodeId. This can be further expanded to have a userId field if you wanted to keep track of what episodes your friends had been watching.

Another idea could be to tag episodes with certain themes or topics like Klingon episodes or meaning of life episodes or Kirk talks a computer to death episodes. Again this is fairly simple table containing TagName and EpisodeId.

To boldly go where no SQL has gone before

Sun, 12 Oct 2014 20:00:45 +0000

My last post proved quite popular so I am wondering if I can combine a post about IT and Star Trek.

Years ago I used to have lists of Star Trek episodes, which included such information like original air date, production number, episode title and brief description.

One thing that was hard to keep track of was how many episodes were written by a specific person. This is because episodes are written by multiple people. A column called writer would then need to contain multiple people, another option would be to have columns called writer1, writer2 etc. This wouldn’t help either as you wouldn’t know which column a specific writer had been saved in.

The relationship between writer and episode is known as a many to many relationship. An episode can have many writers and a writer can have written many episodes. To achieve this structure in a SQL database you will need three database tables as it is not possible to create a many to many join between two tables. The first table will contain all the episodes, the second table will contain all the writers, the third table known as a junction table, will contain the relationship between the two.

Let’s do an example so we can see how this would work. Gene Roddenberry creator of Star Trek wrote the pilot episode ‘The Cage’. So Gene would be added to the writers table with an id of 1 and The Cage would be added to the episode table with an id of 1. In the junction table, it has two columns episode and writer, so we would enter 1 and 1 into these columns.

Select * from Episode e
Join EpisodeWriter ew on e.id = ew.EpisodeId
Join Writer w on w.id = ew.WriterId

But if Gene Coon and Gene Roddenberry had writing credits on The Cage we would need to add Gene Coon to the writing table and add an extra entry to the episodewriter table.

I am going to do more posts based on this as I expand the database structure to include other information, I may go on to create stored procedures for bringing back certain information or I may use this as an example to talk about coding a user interface.

Destination Star Trek

Sun, 05 Oct 2014 20:00:45 +0000

As a break from my usual topics I am going to talk about what I did over the weekend.

I am a huge star trek fan, so me and my wife Laura went to Destination Star Trek, a convention held in London. This is a picture of us in costume, my wife created a tribble costume (A tribble is an alien race that just consists of a ball of fur) and I am wearing an original series captains uniform. We had a great time.

There are several things that made this trip special.

Friends: After the last convention I remember thinking how much better the convention would be with a group of friends. Shortly afterwards I started listening to the trekmate podcast, eventually this lead to me helping out with their website and making friends via twitter with many of the hosts. This weekend was the first time I met up with them and it was great to put faces to names (or twitter names)
Writers and Directors: Star Trek has a great philosophy. In the future human beings will put their differences aside and work together to explore the galaxy, this is one reason why Star Trek is so popular even after almost 50 years. This philosophy has been created by the writers and other creative staff over the years. My favourite Star Trek film is Wrath of Khan which was directed by Nicholas Meyer, this weekend I got my DVD signed by him and was able to tell him how I still enjoyed that film over 30 years after it was made. He said that he really appreciated me saying that. Listening to his talk later on, it was fascinating to hear his insights into making my favourite film. It is the writers and directors that make our favourite characters who they are, the actors themselves aren’t allowed to ad lib, all they do is flesh them out with a bit of emotion or a mannerism, it is the writer that create our favourite lines or situations that we remember.
Seven of Nine: During my teenage years I had lots of posters of Seven on my walls. Seven of Nine was the ex-borg that walked around the ship in a skin tight catsuit, but she was also highly intelligent and continued to explore what it was to be human. It was great to have my photo taken with her, even if it was rushed due to the large number of people that wanted their picture taken.
Trivia: On the Friday of the convention, Laura took part in the beginners trivia challenge, she won. On Saturday I took part in the intermediate trivia challenge, I did really badly only got 3 correct, On Sunday I took part in the captain level challenge and I won. I really wasn’t expecting that, after I did so badly the day before. Thanks Marc Stamper and Destination Star Trek for organising that it was great fun, and also fun to meet the other trekkies who I had been competing against via twitter.
Party!: On Saturday night we went to a party, we had Romulan Ale to drink, we had some of the actors performing songs (James Darren was particularly good), and some of the actors mingled with fans, it was great to see the actors relaxing and if you were lucky you could chat to them, I asked Vaughn Armstrong how he was enjoying the music, he loved it.

This is just a taste of what I got up to, there was lots more, like seeing alien costumes, listening to talks, sitting on a mockup of the bridge. Only one way to end this post, Live Long and Prosper!

Windows 9 or do I mean 10

Wed, 01 Oct 2014 20:00:45 +0000

Like many systems administrators I watched as the news came out yesterday about the latest version of windows. What would it be like? What would it be called? Would it be better or worse than previous versions?

I was shocked when the name of the new operating system was revealed to be Windows 10. What really!? At first I though someone had tweeted a photoshopped image of the press conference, but no it was really going to be called Windows 10.

Windows 10 is the next operating system to be released after windows 8, many had been calling it windows 9 or windows Threshold.

Microsoft has never had a fixed naming convention for its OSes, so should we really be that surprised at its new name.

Windows 1.0 (1985)
Windows 2.0 (1987)
Windows 3.0 (1990)
Windows 95 (1995)
Windows 98 (1998)
Windows ME (2000)
Windows XP (2001)
Windows Vista (2007)
Windows 7 (2009)
Windows 8 (2012)
Windows 10 (2015)
Windows 11 (2021)

One reason Microsoft may have gone with with Windows 10 is because they wanted to signify that the coming Windows release would be the last “major” Windows update. What does it mean to be the last major update?

If you look at the list of OSes above you will see that between XP and Vista was a huge 6 years, this gap is/has caused a huge headache for IT professionals (me included). Software was written for XP because that is all that was available, when XP moved out of support earlier this year it caused loads as problems as people tried to get this software to work on more modern OSes.

If windows 10 is around for a long time we may end up with a similar situation to XP, in that loads of software will run only on it and cause problems if you try and port it to whatever comes next. I am only a trainee developer so I don’t know what is involved with building an OS, but I would imagine there must be things that are very difficult or almost impossible to fix without ripping them out and starting again which is what happens when a new version of an OS is written.

Only time will tell if Windows 10 really is the last version of windows and what will happen to the OS in the future. What ever the case I am downloading a preview so I can have a look (I want my start menu back!)

What is the difference between Development and Operations?

Sat, 27 Sep 2014 20:00:45 +0000

I want to learn Development and eventually move into a development role. But what is the difference between development and what I already do?

Put simply Development or programming is the creation of new programs, websites or databases.

IT Operations or System Administration is the administration of existing servers, websites or databases.

I have been doing System Administration since I started my job back in 2006. What you do as part of this role can be very simple like setting up new users or computers to something very complex like upgrading the version of Exchange that the company uses for its emails.

As part of my role I have done a lot of tasks that could be described as development. I have created databases and built new database structures like tables, views and stored procedures. I have also assisted the development department with testing, setting up visual studio, building websites and databases from code.

Can I describe myself as a developer? I would say yes, I have done plenty of work that is development work.

Can I get a job as a developer? Possibly but it would depend on the role. I have many of the skills that developers use, but I have limited knowledge in many areas. What I need to do is concentrate my efforts onto the development work I do and delegate as much administration work as I can so my knowledge can increase.

A Buzz word in IT at the moment is DevOps. I have lost count of the number of times .NetRocks or RunAsRadio have mentioned it.

DevOps is the integration of IT Development and Operations, it emphasises the need for the two departments to work closely together to achieve common goals. In big companies the two departments can pull against each other if you are not careful, but in my case as I do both roles, it would be very difficult for the development side of myself to blame the operations side of myself for a problem.

This I think puts me in a good position, I just need to learn more development and I will make a valuable addition to someone’s development team.

I love Nagios

Wed, 24 Sep 2014 20:00:45 +0000

You may not have heard of Nagios but it has saved my bacon quite a few times.

Nagios is an open source server monitoring application that runs on many linux flavours.

I can’t remember exactly when I first installed nagios but I am guessing it was sometime in 2007/8. My boss gave me a book about it (which I never read) and told me to create a system to monitor the companies servers.

Nagios is not simple to set up. It relies on setting up various Hosts and services. Hosts are usually physical servers that you want to monitor and services are all the services you want to monitor. As this is a linux program all these can be configured by editing the right config file

Nagios is very flexible and can be expanded easily with the use of plugins, if you want to monitor something there is usually a plugin available. If you have a dell server running openmanage software there is even a plugin that allows the temperature of your server to be monitored.

If you want to monitor windows servers the use of nsclient++ is a real advantage. This is a simple client that runs as a service on your windows server. This allows nagios to track memory, cpu, disk space, performance and services, in fact almost everything that you would want to monitor.

Over the years I have kept a close eye on Nagios and added extra checks as new services were added or problems encountered. A few years ago I dabbled with sending alerts out via SMS message and once I got a smart phone found an app to keep track of Nagios 24/7.

But recently I have started wondering if Nagios is the best way to monitor modern servers like 2012 or remote services like Azure. I want something that is easy to expand as your IT infrastructure expands. Something that relies on running on a linux OS requires your IT staff have a knowledge of linux and you keep that server maintained and updated.

My question is: Is Nagios still the best way to monitor my servers?

About Me

Mon, 01 Jan 0001 00:00:00 +0000

My real name is Simon Foster but my on-line persona is Funky Si a nickname I was given during my university days which has somehow stuck. I am a Developer working in the North of England. I have been working in IT departments since 2006 and have a wide range of experiences. More recently in the Software Development space but also SysAdmin, Server Infrastructure, Databases and DevOps.

In my spare time I have created Pwned Pass a Xamarin Forms mobile app that makes use of the Have I Been Pwned? API to allow searching for breached emails and passwords. This is available from the Google Play store.

Certifications

Tech I have used

.Net
C#
SQL Server
Blazor
Javascript
jquery
Azure
Azure DevOps
AWS
Terraform
Powershell
git
plus many others…

Recommendations

Simon is an excellent developer, always looking for new methods and technology to improve existing solutions, along with his own skills. He is open to new ideas, though not afraid to share his thoughts and feedback and guide others to the best solution possible. It’s worth noting that Simon has vast knowledge on hardware and infrastructure which really does compliment his skill as a developer. - Alasdair Thomson (Head of Business Data Solutions at NPD Travel Retail)

Simon worked for my company for a number of years and was a very valued member of staff, and later on management. He is an intelligent and conscientious worker who gives over and above on a regular basis. He leads by example and was respected by his peers and the directors of the company. I would not hesitate to recommend Simon both in terms of the quality of his work and his ethics and values in life in general. - Alison Davies (CEO at Eurosafe UK)

If you want a developer who really understands DevOps, then Si is your man. With a mixture of operations, support, development and management experience, he can bring a balanced skillset to a team, as well as excellent knowledge of Azure, .NET, SQL and the web. - Keith Williams (Development Manager at IRIS for Cascade HR and KashFlow Payroll)

Events

Mon, 01 Jan 0001 00:00:00 +0000

A list of events I have attended.

Event/Talk	Date	Location
	2024
Scottish Summit	17/19 October 2024	Aberdeen
SteelCon	20 July 2024	Sheffield
DDD North	2 March 2024	Hull
	2023
DevBoss	16 November 2023	Leeds
DDD East Midlands	7th October	Nottingham
SteelCon	8 July 2023	Sheffield
Yorkshire Azure	13 June 2023	Leeds
DevBoss	20 April 2023	Leeds
dotnet York Machine Learning with ML.Net	2 March 2023	York
	2022
Microsoft Datacenter Tour: Virtual Experience	6 December 2022	Online
DDD North	3 December 2022	Hull
LeedsSharp Azure Patterns for Fun and Profit	24 November 2022	Leeds
.Net Conf (Keynote only)	8 November 2022	Online
Scottish Summit	10/11 June 2022	Glasgow
The .Network (Networking event)	28 April 2022	Leeds

Funky Si: The Next Generation

Mon, 01 Jan 0001 00:00:00 +0000

On 19th August at 9.39 pm James David Martin Foster arrived weighing 9 lb 12.

My initial thoughts on fatherhood can be found here . More baby photos can also be found on my instragram .

On 11th November 2017 at 22:58pm Edward Leonard Alan Foster arrived weighing 11 lb

Podcasts

Mon, 01 Jan 0001 00:00:00 +0000

A list of podcast I have recorded.

TrekRanks #174

Picard Season 3

Temporal Trek Podcast

Brothers, TNG S4 E3 Review, The Battle Bridge

Star Trek is back (in 2017)

The Hunted, TNG S3 E11, The Battle Bridge

Upper Pylon 2 – 1 x 09: The Passenger

Ten Forward Episode #135 – Anti Firbob is Back or Simon’s Desert Island Trek

Privacy Policy

Mon, 01 Jan 0001 00:00:00 +0000

Privacy Policy

Last updated: March 31, 2022

This Privacy Policy describes Our policies and procedures on the collection, use and disclosure of Your information when You use the Service and tells You about Your privacy rights and how the law protects You.

We use Your Personal data to provide and improve the Service. By using the Service, You agree to the collection and use of information in accordance with this Privacy Policy. This Privacy Policy has been created with the help of the Privacy Policy Template.

Interpretation and Definitions

Interpretation

The words of which the initial letter is capitalized have meanings defined under the following conditions. The following definitions shall have the same meaning regardless of whether they appear in singular or in plural.

Definitions

For the purposes of this Privacy Policy:

Account means a unique account created for You to access our Service or parts of our Service.
Company (referred to as either "the Company", "We", "Us" or "Our" in this Agreement) refers to Funky Si's Blog.
Cookies are small files that are placed on Your computer, mobile device or any other device by a website, containing the details of Your browsing history on that website among its many uses.
Country refers to: United Kingdom
Device means any device that can access the Service such as a computer, a cellphone or a digital tablet.
Personal Data is any information that relates to an identified or identifiable individual.
Service refers to the Website.
Service Provider means any natural or legal person who processes the data on behalf of the Company. It refers to third-party companies or individuals employed by the Company to facilitate the Service, to provide the Service on behalf of the Company, to perform services related to the Service or to assist the Company in analyzing how the Service is used.
Usage Data refers to data collected automatically, either generated by the use of the Service or from the Service infrastructure itself (for example, the duration of a page visit).
Website refers to Funky Si's Blog, accessible from https://www.funkysi1701.com
You means the individual accessing or using the Service, or the company, or other legal entity on behalf of which such individual is accessing or using the Service, as applicable.

Collecting and Using Your Personal Data

Types of Data Collected

Personal Data

While using Our Service, We may ask You to provide Us with certain personally identifiable information that can be used to contact or identify You. Personally identifiable information may include, but is not limited to:

Email address
First name and last name
Usage Data

Usage Data

Usage Data is collected automatically when using the Service.

Usage Data may include information such as Your Device's Internet Protocol address (e.g. IP address), browser type, browser version, the pages of our Service that You visit, the time and date of Your visit, the time spent on those pages, unique device identifiers and other diagnostic data.

When You access the Service by or through a mobile device, We may collect certain information automatically, including, but not limited to, the type of mobile device You use, Your mobile device unique ID, the IP address of Your mobile device, Your mobile operating system, the type of mobile Internet browser You use, unique device identifiers and other diagnostic data.

We may also collect information that Your browser sends whenever You visit our Service or when You access the Service by or through a mobile device.

Tracking Technologies and Cookies

We use Cookies and similar tracking technologies to track the activity on Our Service and store certain information. Tracking technologies used are beacons, tags, and scripts to collect and track information and to improve and analyze Our Service. The technologies We use may include:

Cookies or Browser Cookies. A cookie is a small file placed on Your Device. You can instruct Your browser to refuse all Cookies or to indicate when a Cookie is being sent. However, if You do not accept Cookies, You may not be able to use some parts of our Service. Unless you have adjusted Your browser setting so that it will refuse Cookies, our Service may use Cookies.
Flash Cookies. Certain features of our Service may use local stored objects (or Flash Cookies) to collect and store information about Your preferences or Your activity on our Service. Flash Cookies are not managed by the same browser settings as those used for Browser Cookies. For more information on how You can delete Flash Cookies, please read "Where can I change the settings for disabling, or deleting local shared objects?" available at https://helpx.adobe.com/flash-player/kb/disable-local-shared-objects-flash.html#main_Where_can_I_change_the_settings_for_disabling__or_deleting_local_shared_objects_
Web Beacons. Certain sections of our Service and our emails may contain small electronic files known as web beacons (also referred to as clear gifs, pixel tags, and single-pixel gifs) that permit the Company, for example, to count users who have visited those pages or opened an email and for other related website statistics (for example, recording the popularity of a certain section and verifying system and server integrity).

Cookies can be "Persistent" or "Session" Cookies. Persistent Cookies remain on Your personal computer or mobile device when You go offline, while Session Cookies are deleted as soon as You close Your web browser. Learn more about cookies: Use of Cookies by Free Privacy Policy.

We use both Session and Persistent Cookies for the purposes set out below:

Necessary / Essential Cookies

Type: Session Cookies

Administered by: Us

Purpose: These Cookies are essential to provide You with services available through the Website and to enable You to use some of its features. They help to authenticate users and prevent fraudulent use of user accounts. Without these Cookies, the services that You have asked for cannot be provided, and We only use these Cookies to provide You with those services.
Cookies Policy / Notice Acceptance Cookies

Type: Persistent Cookies

Administered by: Us

Purpose: These Cookies identify if users have accepted the use of cookies on the Website.
Functionality Cookies

Type: Persistent Cookies

Administered by: Us

Purpose: These Cookies allow us to remember choices You make when You use the Website, such as remembering your login details or language preference. The purpose of these Cookies is to provide You with a more personal experience and to avoid You having to re-enter your preferences every time You use the Website.

For more information about the cookies we use and your choices regarding cookies, please visit our Cookies Policy or the Cookies section of our Privacy Policy.

Use of Your Personal Data

The Company may use Personal Data for the following purposes:

To provide and maintain our Service, including to monitor the usage of our Service.
To manage Your Account: to manage Your registration as a user of the Service. The Personal Data You provide can give You access to different functionalities of the Service that are available to You as a registered user.
For the performance of a contract: the development, compliance and undertaking of the purchase contract for the products, items or services You have purchased or of any other contract with Us through the Service.
To contact You: To contact You by email, telephone calls, SMS, or other equivalent forms of electronic communication, such as a mobile application's push notifications regarding updates or informative communications related to the functionalities, products or contracted services, including the security updates, when necessary or reasonable for their implementation.
To provide You with news, special offers and general information about other goods, services and events which we offer that are similar to those that you have already purchased or enquired about unless You have opted not to receive such information.
To manage Your requests: To attend and manage Your requests to Us.
For business transfers: We may use Your information to evaluate or conduct a merger, divestiture, restructuring, reorganization, dissolution, or other sale or transfer of some or all of Our assets, whether as a going concern or as part of bankruptcy, liquidation, or similar proceeding, in which Personal Data held by Us about our Service users is among the assets transferred.
For other purposes: We may use Your information for other purposes, such as data analysis, identifying usage trends, determining the effectiveness of our promotional campaigns and to evaluate and improve our Service, products, services, marketing and your experience.

We may share Your personal information in the following situations:

With Service Providers: We may share Your personal information with Service Providers to monitor and analyze the use of our Service, to contact You.
For business transfers: We may share or transfer Your personal information in connection with, or during negotiations of, any merger, sale of Company assets, financing, or acquisition of all or a portion of Our business to another company.
With Affiliates: We may share Your information with Our affiliates, in which case we will require those affiliates to honor this Privacy Policy. Affiliates include Our parent company and any other subsidiaries, joint venture partners or other companies that We control or that are under common control with Us.
With business partners: We may share Your information with Our business partners to offer You certain products, services or promotions.
With other users: when You share personal information or otherwise interact in the public areas with other users, such information may be viewed by all users and may be publicly distributed outside.
With Your consent: We may disclose Your personal information for any other purpose with Your consent.

Retention of Your Personal Data

The Company will retain Your Personal Data only for as long as is necessary for the purposes set out in this Privacy Policy. We will retain and use Your Personal Data to the extent necessary to comply with our legal obligations (for example, if we are required to retain your data to comply with applicable laws), resolve disputes, and enforce our legal agreements and policies.

The Company will also retain Usage Data for internal analysis purposes. Usage Data is generally retained for a shorter period of time, except when this data is used to strengthen the security or to improve the functionality of Our Service, or We are legally obligated to retain this data for longer time periods.

Transfer of Your Personal Data

Your information, including Personal Data, is processed at the Company's operating offices and in any other places where the parties involved in the processing are located. It means that this information may be transferred to — and maintained on — computers located outside of Your state, province, country or other governmental jurisdiction where the data protection laws may differ than those from Your jurisdiction.

Your consent to this Privacy Policy followed by Your submission of such information represents Your agreement to that transfer.

The Company will take all steps reasonably necessary to ensure that Your data is treated securely and in accordance with this Privacy Policy and no transfer of Your Personal Data will take place to an organization or a country unless there are adequate controls in place including the security of Your data and other personal information.

Disclosure of Your Personal Data

Business Transactions

If the Company is involved in a merger, acquisition or asset sale, Your Personal Data may be transferred. We will provide notice before Your Personal Data is transferred and becomes subject to a different Privacy Policy.

Law enforcement

Under certain circumstances, the Company may be required to disclose Your Personal Data if required to do so by law or in response to valid requests by public authorities (e.g. a court or a government agency).

Other legal requirements

The Company may disclose Your Personal Data in the good faith belief that such action is necessary to:

Comply with a legal obligation
Protect and defend the rights or property of the Company
Prevent or investigate possible wrongdoing in connection with the Service
Protect the personal safety of Users of the Service or the public
Protect against legal liability

Security of Your Personal Data

The security of Your Personal Data is important to Us, but remember that no method of transmission over the Internet, or method of electronic storage is 100% secure. While We strive to use commercially acceptable means to protect Your Personal Data, We cannot guarantee its absolute security.

Children's Privacy

Our Service does not address anyone under the age of 13. We do not knowingly collect personally identifiable information from anyone under the age of 13. If You are a parent or guardian and You are aware that Your child has provided Us with Personal Data, please contact Us. If We become aware that We have collected Personal Data from anyone under the age of 13 without verification of parental consent, We take steps to remove that information from Our servers.

If We need to rely on consent as a legal basis for processing Your information and Your country requires consent from a parent, We may require Your parent's consent before We collect and use that information.

Links to Other Websites

Our Service may contain links to other websites that are not operated by Us. If You click on a third party link, You will be directed to that third party's site. We strongly advise You to review the Privacy Policy of every site You visit.

We have no control over and assume no responsibility for the content, privacy policies or practices of any third party sites or services.

Changes to this Privacy Policy

We may update Our Privacy Policy from time to time. We will notify You of any changes by posting the new Privacy Policy on this page.

We will let You know via email and/or a prominent notice on Our Service, prior to the change becoming effective and update the "Last updated" date at the top of this Privacy Policy.

You are advised to review this Privacy Policy periodically for any changes. Changes to this Privacy Policy are effective when they are posted on this page.

Contact Us

If you have any questions about this Privacy Policy, You can contact us:

By email: simon@funkysi1701.com

Pwned Passwords

Mon, 01 Jan 0001 00:00:00 +0000

Using the data supplied by Troy Hunt and his Have I been pwned? website Pwned Pass allows you to check to see if any password has appeared in a data breach.

For more details about Have I been pwned? check out haveibeenpwned.com and www.troyhunt.com.

Pwned Pass is a simple Xamarin app that allows you to type in a password and tells you if it has been used in a data breach.

Troy Hunt of Have I Been Pwned? recently added a new API to his website which allows you to search his extensive database of pwned passwords, over 306 million of them. I have simply created a Android frontend to this API.

It should be noted: Do not send any password you actively use to a third-party service – even mine! I don’t log anything that you type into my app and this app makes use of the k-anonymity feature to avoid transmitting passwords.

As well as using Troy’s new API I also take advantage of his existing API’s. You can search his extensive database of email addresses to see if you have been affected by a data breach all from my android app.

Pwned Pass is now available from the Google Play Store.