Search

Coding and stuff

Just another WordPress.com site

Submodules



git clone --recursive git://github.com/foo/bar.git

in submodule directory
git submodule update --init --recursive

updating the submodule:
remember to commit / push the submodule first. 
the commit / push the containing repo

Setting up (more tech) at home

I’ve recently had an itch that I wanted more digital stuff in my life…

  • A better digital backup solution (too many photos to loose). Some kind of NAS device.
  • I wanted an always on computer to access media from.
  • Possibly host a website (well, the computer is on anyway!).
  • I also wanted to try the Ubuntu server.
  • I wanted to build another PC

So to start with, I was looking at NAS devices – Synology or ASUSTOR or a homebuild. Doing some research into the purpose NAS devices quickly showed that they are not as easy to configure as they should be, and thing come with almost no memory (I spoke to my brother about this recently, and he had given up trying to get his Synology to do what he wanted). One article I read suggested that there should be about 1GB or memmory for each TB of hard disk space. I recently bought a “cheap” seagate custom NAS box, and it was so dreadfully slow, that I know that I would need something more powerful.

So you can buy a QSNAP TVS-471 NAS for just under £1000 – and that is a lot of money for an unpopulated NAS (i.e. no hard drives included). Now it may be that this would have been the perfect solution, but I wouldn’t have had the fun or building my own machine, and nor picking my own components. The i3 included is an older generation, and has a TPD or 54 W. I could buy a next gen i3 with a TPD of just 35W.

Anyway, I have gone the home build route for the moment.

i3-6100, 16 GB RAM, micro-atx asus, M.2 PCI-SSD, 2*5GB drives, small gold rated powersupply. This ended up half the price of an “off the shelf” product.

I’m a complete Noob when it comes to Lynux. I created a virtual box and installed ubuntu server. I thought about things like FreeNas, but decided to go with the server edition to start with. This worked reasonably well, but I could get the networking bridging working as I wanted – this could have been many things, but though I was able to access the external network, and did have some issues with SSH (openssh server) and remote admin.

So after having got this “working” in a VM, I installed on my new local machine. I made the mistake of allowing the drive to be encrypted (not really a mistake – more of a pain in the ass when you’re doing lots or reboots – especially with a long password).

OK, so I’ve got a running server, which is able to see the outside world, and everything is going swimmingly. I got samba loaded and configured (but just on the primary m.2 ssd. I did a server “software” RAID 1 on the 2 * 5 GB drives, and got these working with samba too. I was able to save, and access files from other locations in my home.

I then tried to access the media from my modern mid-range smart TV (to play music through  – a LG770). The TV didn’t see the media. A little research later, and looking at the dlna standard I installed MediaTomb. Yay – that installed simply, and I managed to get it to look at the local drive (and I had a small amount of media copied there to practice with). However, I couldn’t get it to see my separate HDDs. On a separate note, I had given up trying to get the RAID working within ubuntu – I know that it can be done, but I had depriorised that whilst looking at other issues.

So I started to build this machine at about 11am, and had it physically finished early afternoon – 2pm. Basic ubuntu server installed with stuff like samba configured by about 4pm. I spent several hours with the raid, media tomb, and samba. I installed the ubuntu desktop at some stage in the afternoon, as it made some thing much easier. I got a bit annoyed with stuff by about midnight, and decided to try installed Windows 10, and see how that worked (remember that ubuntu is free, and windows server is £100).

I downloaded and installed a copy of window (I did call the Microsoft shop and they advised that the download could not be installed by USB key). I did a quick download of windows anyway (from the official MS site), but havn’t activated yet, as I want to check that it’ll do what I want it to.

I got windows working, with the RAID setup in under an hour. The TV didn’t see the media, however. I also discovered a “feature” of windows called “Homegroup”. The key to get the media working with the pc was found http://www.wikihow.com/Connect-PC-to-LG-Smart-TV   – basically just add media player. I did also see that Plex server was also visible, but didn’t have time to further investigate.

I’ve not got the website working yet – though I know how to do that from an IIS standpoint – I’m not sure about the DNS, so I’ll need to do some investigation with my ISP. I’m not sure whether this is worth the added security risk.

It was a fun way to spend a couple of days holiday. I’m still not 100% there. I need to make an end decision whether to stay with Windows or put ubuntu server (or freeNas) back on – If I just install directly to the raid and ignore the ssd that would make life easier. I certainly found the Windows side easier, however I’m not yet convinced that its worth the additional cost. There are, however, a number of questions that I still haven’t answered about ubuntu though – for example – if I encrypt the primary drive, then does this mean that whenever I reboot the machine I need to log  back into the system. Ubuntu is amazing really. There is a lot to like about it, however there are a lot of thing that are just annoying – just choosing a text editor – vi rather than nano or  gedit.

 

DNVM

setup an alias

dnvm alias default486 1.0.0-beta4 -arch x86

use the alias

dnvm use default486 -p

DbContext Error

To get detailed information about errors that the DbContext Save has raised: 


public static class DataExtensions
{
public static int SaveChangesWithErrors(this DbContext context)
{
try
{
return context.SaveChanges();
}
catch (DbEntityValidationException ex)
{
StringBuilder sb = new StringBuilder();


foreach (var failure in ex.EntityValidationErrors)
{
sb.AppendFormat("{0} failed validation\n", failure.Entry.Entity.GetType());
foreach (var error in failure.ValidationErrors)
{
sb.AppendFormat("- {0} : {1}", error.PropertyName, error.ErrorMessage);
sb.AppendLine();
}
}

throw new DbEntityValidationException(
"Entity Validation Failed - errors follow:\n" +
sb.ToString(), ex
); // Add the original exception as the innerException
}
}
}

Media Formatter – MVC 6

I saw a post that said that the MediaTypeFormatter has now moved to IInputFormatter and IOutputFormatter

There’s a link to an example of someone (rynowak) using an output formatter here:

https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNet.Mvc.Core/Formatters/StringOutputFormatter.cs

Visual studio 2015, ASP.net 5, MVC6, c# 6 … initial views and the 30 second tour! …

I’ve spent a little time with VS 2015 (RC) now. There is HUGE amount that has changed.

c# 6 … 

For me the biggest win here is (and this is a big win):

Null conditional operators…

customer?.Orders

A list of some of the changes can be found here:

https://github.com/dotnet/roslyn/wiki/Languages-features-in-C%23-6-and-VB-14

You can have a quick play with (most of) the features at: https://dotnetfiddle.net/

Visual studio 2015

For me the biggest win here is:

lambda debugging .. and using lambdas in the immediate window. (another huge win)

They (MS) look like they are trying to

ASP.net

(with MVC) bower, gulp, grunt NPM

These packages bring a lot of power, however they also bring a fair mount of learning, and hassle. This means that rather spending 5 minutes putting in a script library, you can now spend an hour doing it!!! That said, you can do cool stuff with it along the way. It just seems to me that this will result in additional technical debt for project initialization.

Publishing… oh dear. Is MS under the mistaken belief that developers actually use the publish feature. Do they not appreciate CI? I guess that I’ll work this out eventually, but at the moment its like wading through treacle.

Huge changes to the syntax, to html tag helpers to, to config files, DI etc.

http://docs.asp.net/en/latest/conceptual-overview/aspnet.html

MVC

Mvc now coexists with webapi – cool

No more “goto view” – how dissapointing. (but possibly because of coexisting with webapi?)

No more “Action”. This has been replaced by web components.

Note the comments about bower etc, and publishing from the asp.net section above.

EF

I’ve not looked at the latest version of this yet, but I see that with the upcoming EF7, the intention is to drop EDMX files, with a shift towards code first. I’ve found code first fairly easy to use, however confusion can arise around the data annotations vs fluent api  – read up on the views of Julia Lerman. I don’t really like the “view” implementation. It’s also easy to make a really bad data schema using code first…. creating hundreds of properties with nvarchar(max). I also really don’t like the MS implementation of Many to Many relationships with Entity Framework.

Globbing patterns (things like **)

Globbing patterns

These are used in things like Gulp/Grunt

From the Grunt website:

It is often impractical to specify all source filepaths individually, so Grunt supports filename expansion (also know as globbing) via the built-in node-glob and minimatch libraries.

While this isn’t a comprehensive tutorial on globbing patterns, know that in a filepath:

  • * matches any number of characters, but not /
  • ? matches a single character, but not /
  • ** matches any number of characters, including /, as long as it’s the only thing in a path part
  • {} allows for a comma-separated list of “or” expressions
  • ! at the beginning of a pattern will negate the match

For more information about globs, see:

https://www.npmjs.com/package/glob

http://en.wikipedia.org/wiki/Glob_%28programming%29

Finding foreign keys…

sp_fkeys ‘dataitems’

SQL Collation

Find the collation on a table (useful for excel imports)
SELECT
col.name, col.collation_name
FROM
sys.columns col
WHERE
object_id = OBJECT_ID(‘ImportConcepts’)

Change the collation where necessary:
ALTER TABLE ImportConcepts

ALTER COLUMN Category NVARCHAR(255)

COLLATE Latin1_General_CI_AS NOT NULL

spreadsheets with opendocument format

Xlsx files are basically zip files.

You’ll need to add the following using statements:
using DocumentFormat.OpenXml;
using DocumentFormat.OpenXml.Packaging;
using DocumentFormat.OpenXml.Spreadsheet;

(additionally you’ll need to add references to WindowsBase (for the compression), and DocumentFormat.OpenXML which can be downloaded from the open xml sdk: http://www.microsoft.com/en-gb/download/details.aspx?id=30425)

Basic spreadsheet parsing for the openDocument standard can use with DOM or SAX.


using (SpreadsheetDocument spreadsheetDocument = SpreadsheetDocument.Open(physicalPath, false))
{
WorkbookPart workbookPart = spreadsheetDocument.WorkbookPart;
foreach (WorksheetPart wsp in workbookPart.WorksheetParts)
{
SheetData sheetData = wsp.Worksheet.Elements().First();

foreach (Row row in sheetData.Elements())
{
checkInitialSpans(row);
Score s = new Score();
// workbook needs to be passed through as spreadsheet strings are not stored in the cell, but a separate lookup table.
s.ScoreName = row.getScoreName(workbookPart);
s.ScoreValue = row.getScoreValue();
product.Scores.Add(s);
}
}
}

The biggest thing to notice is that strings are stored in a different location, and you you’ll end up wondering why you always get back “0”.


public static double getScoreValue(this Row r)
{
double cellValue = -1;

Cell c = (Cell)r.ElementAt(1);
cellValue = double.Parse(c.CellValue.Text);
return cellValue;
}

public static string getScoreName(this Row r, WorkbookPart workbookPart)
{
string cellText = string.Empty;
Cell c = (Cell)r.ElementAt(0);
if (c != null && c.DataType != null && c.DataType == CellValues.SharedString)
{
int id = -1;
Int32.TryParse(c.InnerText, out id);
var item = GetSharedStringItemById(workbookPart, id);
if (item.Text != null)
{
cellText = item.Text.Text;
}
else if (item.InnerText != null)
{
cellText = item.InnerText;
}
else if (item.InnerXml != null)
{
cellText = item.InnerXml;
}
}
return cellText;
}

public static SharedStringItem GetSharedStringItemById(WorkbookPart workbookPart, int id)
{
return workbookPart.SharedStringTablePart.SharedStringTable.Elements().ElementAt(id);
}

Accessing via SAX, would be:

using (SpreadsheetDocument spreadsheetDocument = SpreadsheetDocument.Open(product.SystemFileName, false))
{
WorkbookPart workbookPart = spreadsheetDocument.WorkbookPart;
var sheetCount = workbookPart.Workbook.Sheets.Count();
foreach (WorksheetPart wsp in workbookPart.WorksheetParts)
{
var columnCount = wsp.Worksheet.Descendants().Count();
var rowCount = wsp.Worksheet.Descendants().Count();
var cellCount = wsp.Worksheet.Descendants().Count();

OpenXmlReader reader = OpenXmlReader.Create(wsp);
string text;
while (reader.Read())
{
text = reader.GetText();
System.Diagnostics.Debug.WriteLine(text + "");
}
}
}

For more information about these methods, see the example at MSDN: https://msdn.microsoft.com/en-us/library/office/gg575571.aspx

Alternatively there are several thirdparty libraries (like: https://github.com/ExcelDataReader/ExcelDataReader) or there is the ever present OLEDB or SSIS.

Parsing via OLEDB would be something like:
private static DataTable ParseOldDb(string physicalPath)
{
string conStr = "Provider=Microsoft.ACE.OLEDB.12.0;data source='" + physicalPath + "';Extended Properties=\"Excel 12.0;HDR=NO;IMEX=1\" ";
string cmd = "select * from [sheet1$]";
var dt = new DataTable();

using (OleDbConnection cn = new OleDbConnection(conStr))
{
cn.Open();
DataTable schema = cn.GetSchema();

using (OleDbDataAdapter ad = new OleDbDataAdapter(cmd, cn))
{
ad.Fill(dt);
return dt;
}
}
}

Blog at WordPress.com. | The Baskerville Theme.

Up ↑

Follow

Get every new post delivered to your Inbox.